Name : Hyun Shik Chung, Jarvis Dao

ITSC 4156 – Intro to Machine Learning

Using LSTM to Predict Bitcoin Prices:

Introduction

What is Blockchain

By allowing digital information to be distributed but not copied, blockchain technology created the backbone of a new type of internet. Originally devised for the digital currency, Bitcoin, (Buy Bitcoin) the tech community has now found other potential uses for the technology.

Library

In [1]:
from numpy import *
import numpy as np
import pandas as pd
from scipy import stats
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import GRU
from keras.layers import Dropout
from keras.callbacks import EarlyStopping
from keras import initializers
from matplotlib import pyplot
from datetime import datetime
from matplotlib import pyplot as plt
import plotly.offline as py
import plotly.graph_objs as go
from sklearn.preprocessing import MinMaxScaler
%matplotlib inline
Using TensorFlow backend.

What are different about our project.

1. Different dataset

2. LSTM layers

3. loss function

In [2]:
from IPython.display import IFrame

IFrame(src='https://coinmetrics.io/data-downloads/', width=900, height=400)
Out[2]:
In [3]:
dataset = pd.read_csv('ltcForTrain.csv')
In [4]:
print ("Data shape : ",dataset.shape)
dataset.head(5)
Data shape :  (2744, 17)
Out[4]:
date txVolume(USD) adjustedTxVolume(USD) txCount marketcap(USD) price(USD) exchangeVolume(USD) realizedCap(USD) generatedCoins fees activeAddresses averageDifficulty paymentCount medianTxValue(USD) medianFee blockSize blockCount
0 2011-10-07 NaN NaN NaN NaN NaN NaN 0.0 NaN NaN NaN NaN NaN NaN NaN NaN NaN
1 2011-10-08 NaN NaN 0.0 NaN NaN NaN 0.0 50.0 0.0 1.0 0.000244 0.0 NaN NaN 215.0 1.0
2 2011-10-09 NaN NaN 0.0 NaN NaN NaN 0.0 NaN 0.0 0.0 NaN 0.0 NaN NaN 0.0 0.0
3 2011-10-10 NaN NaN 0.0 NaN NaN NaN 0.0 NaN 0.0 0.0 NaN 0.0 NaN NaN 0.0 0.0
4 2011-10-11 NaN NaN 0.0 NaN NaN NaN 0.0 NaN 0.0 0.0 NaN 0.0 NaN NaN 0.0 0.0

There are 569 missing value.

In [5]:
dataset.isnull().sum()
Out[5]:
date                       0
txVolume(USD)            569
adjustedTxVolume(USD)    569
txCount                    1
marketcap(USD)           569
price(USD)               569
exchangeVolume(USD)      569
realizedCap(USD)           0
generatedCoins             4
fees                       1
activeAddresses            1
averageDifficulty          4
paymentCount               1
medianTxValue(USD)       569
medianFee                  6
blockSize                  1
blockCount                 1
dtype: int64
In [6]:
dataset.describe()
Out[6]:
txVolume(USD) adjustedTxVolume(USD) txCount marketcap(USD) price(USD) exchangeVolume(USD) realizedCap(USD) generatedCoins fees activeAddresses averageDifficulty paymentCount medianTxValue(USD) medianFee blockSize blockCount
count 2.175000e+03 2.175000e+03 2743.000000 2.175000e+03 2175.000000 2.175000e+03 2.744000e+03 2740.000000 2743.000000 2743.000000 2.740000e+03 2.743000e+03 2175.000000 2738.000000 2.743000e+03 2743.000000
mean 2.194791e+08 6.855580e+07 11355.643821 1.696205e+09 31.892018 2.033175e+08 1.367312e+09 22379.715596 55.693079 35086.993802 1.420050e+06 2.827418e+04 173.553970 0.004611 7.024114e+06 588.004375
std 6.439949e+08 1.849340e+08 18226.614984 2.957302e+09 52.805824 4.867494e+08 2.306636e+09 12075.061827 96.677048 53077.073534 3.024695e+06 7.237327e+04 277.453356 0.018874 9.982084e+06 191.230663
min 4.484505e+05 3.792622e+05 0.000000 3.801208e+07 1.150000 0.000000e+00 0.000000e+00 50.000000 0.000000 0.000000 2.441406e-04 0.000000e+00 0.001760 0.000000 0.000000e+00 0.000000
25% 7.930029e+06 3.696041e+06 2972.500000 1.349529e+08 3.355000 1.643170e+06 4.132866e+07 14550.000000 14.972662 8910.500000 9.575075e+02 6.146500e+03 41.363134 0.000224 2.150187e+06 554.000000
50% 3.007672e+07 1.086806e+07 4503.000000 1.985032e+08 4.640000 4.154880e+06 2.801397e+08 19800.000000 25.982844 14007.000000 4.491421e+04 1.099600e+04 87.967825 0.001000 3.384537e+06 583.000000
75% 1.265453e+08 4.638918e+07 14886.000000 2.394446e+09 44.100000 2.595530e+08 6.537123e+08 29300.000000 56.499317 42607.000000 2.294769e+05 3.104450e+04 219.318384 0.001000 8.746815e+06 613.000000
max 1.175670e+10 2.535597e+09 225860.000000 1.953073e+10 359.130000 6.961680e+09 7.313980e+09 484100.000000 1283.861162 607832.000000 1.187950e+07 1.859479e+06 4586.266455 0.100000 1.235592e+08 9682.000000
In [7]:
import seaborn as sns
plt.figure(figsize =(10, 10))
corr = dataset.corr()
sns.heatmap(corr, xticklabels=corr.columns.values ,yticklabels=corr.columns.values)
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-7-92a6fbe91511> in <module>
----> 1 import seaborn as sns
      2 plt.figure(figsize =(10, 10))
      3 corr = dataset.corr()
      4 sns.heatmap(corr, xticklabels=corr.columns.values ,yticklabels=corr.columns.values)

ModuleNotFoundError: No module named 'seaborn'

Cleaning data

Dealing with missing data

In [8]:
#drop row 1~570
dataset.drop(dataset.index[:570], inplace=True)
print(dataset.shape)
dataset.isnull().sum()
(2174, 17)
Out[8]:
date                     0
txVolume(USD)            0
adjustedTxVolume(USD)    0
txCount                  0
marketcap(USD)           0
price(USD)               0
exchangeVolume(USD)      0
realizedCap(USD)         0
generatedCoins           0
fees                     0
activeAddresses          0
averageDifficulty        0
paymentCount             0
medianTxValue(USD)       0
medianFee                0
blockSize                0
blockCount               0
dtype: int64

Drop original index and add new index start from 0

In [9]:
dataset= dataset.reset_index(drop=True)
In [10]:
dataset.head(5)
Out[10]:
date txVolume(USD) adjustedTxVolume(USD) txCount marketcap(USD) price(USD) exchangeVolume(USD) realizedCap(USD) generatedCoins fees activeAddresses averageDifficulty paymentCount medianTxValue(USD) medianFee blockSize blockCount
0 2013-04-29 3.647810e+07 2.111820e+06 9275.0 7.521684e+07 4.37 0.0 2.284118e+07 32500.0000 634.409741 18395.0 437.937821 9542.0 181.679890 0.10 4977931.0 650.0
1 2013-04-30 4.039166e+07 1.969543e+06 9099.0 7.574233e+07 4.40 0.0 2.319259e+07 31350.0000 792.170373 17810.0 437.937821 9301.0 167.863647 0.10 5349282.0 627.0
2 2013-05-01 7.637420e+07 4.691922e+06 8990.0 7.406414e+07 4.29 0.0 2.458101e+07 31699.9795 639.972367 16991.0 471.122764 9326.0 177.125873 0.10 4463820.0 634.0
3 2013-05-02 1.163151e+07 2.501720e+06 8031.0 6.537939e+07 3.78 0.0 2.479254e+07 26150.0000 528.803594 15769.0 482.512511 8269.0 155.490420 0.10 4088911.0 523.0
4 2013-05-03 4.632241e+06 1.964664e+06 6280.0 5.876169e+07 3.39 0.0 2.475425e+07 19900.0000 375.729091 12956.0 482.512511 6519.0 105.126185 0.05 3382172.0 398.0
In [11]:
dataset = dataset.sort_index(axis=1 ,ascending=True)
dataset = dataset.iloc[::-1]
dataset = dataset.sort_index(ascending=True, axis=0)
dataset = dataset.reindex(index = dataset.index[::-1])
dataset = dataset.reset_index()
dataset=dataset.drop('index', axis=1)
dataset.head(5)
Out[11]:
activeAddresses adjustedTxVolume(USD) averageDifficulty blockCount blockSize date exchangeVolume(USD) fees generatedCoins marketcap(USD) medianFee medianTxValue(USD) paymentCount price(USD) realizedCap(USD) txCount txVolume(USD)
0 102834.0 2.727544e+08 1.181510e+07 573.0 65641701.0 2019-04-11 3.461536e+09 17.753439 14325.0 5.418036e+09 0.000221 75.837923 75246.0 88.39 4.632666e+09 26512.0 5.026659e+08
1 99837.0 9.136417e+07 1.187950e+07 557.0 53092124.0 2019-04-10 2.765901e+09 16.292335 13925.0 5.317491e+09 0.000222 66.125885 76006.0 86.77 4.621039e+09 25088.0 1.995761e+08
2 70669.0 8.834103e+07 1.187950e+07 537.0 14846398.0 2019-04-09 2.742631e+09 16.982480 13425.0 5.483492e+09 0.000222 80.628978 46219.0 89.50 4.615411e+09 23994.0 1.891868e+08
3 140714.0 9.404545e+07 1.176827e+07 587.0 123559210.0 2019-04-08 3.295696e+09 18.271835 14675.0 5.656034e+09 0.000226 78.122568 50448.0 92.33 4.607194e+09 25967.0 2.400780e+08
4 162384.0 5.918720e+07 1.141644e+07 611.0 64268502.0 2019-04-07 3.314849e+09 15.560253 15275.0 5.660144e+09 0.000226 69.606270 138655.0 92.42 4.602690e+09 24003.0 1.881442e+08
In [12]:
dataset.shape
Out[12]:
(2174, 17)
In [13]:
py.init_notebook_mode()
btc_trace = go.Scatter(x=dataset['date'], y=dataset['price(USD)'], name= 'Price')
py.iplot([btc_trace])

Drop date

In [14]:
dataset=dataset.drop('date', axis=1)
In [15]:
dataset.head(5)
Out[15]:
activeAddresses adjustedTxVolume(USD) averageDifficulty blockCount blockSize exchangeVolume(USD) fees generatedCoins marketcap(USD) medianFee medianTxValue(USD) paymentCount price(USD) realizedCap(USD) txCount txVolume(USD)
0 102834.0 2.727544e+08 1.181510e+07 573.0 65641701.0 3.461536e+09 17.753439 14325.0 5.418036e+09 0.000221 75.837923 75246.0 88.39 4.632666e+09 26512.0 5.026659e+08
1 99837.0 9.136417e+07 1.187950e+07 557.0 53092124.0 2.765901e+09 16.292335 13925.0 5.317491e+09 0.000222 66.125885 76006.0 86.77 4.621039e+09 25088.0 1.995761e+08
2 70669.0 8.834103e+07 1.187950e+07 537.0 14846398.0 2.742631e+09 16.982480 13425.0 5.483492e+09 0.000222 80.628978 46219.0 89.50 4.615411e+09 23994.0 1.891868e+08
3 140714.0 9.404545e+07 1.176827e+07 587.0 123559210.0 3.295696e+09 18.271835 14675.0 5.656034e+09 0.000226 78.122568 50448.0 92.33 4.607194e+09 25967.0 2.400780e+08
4 162384.0 5.918720e+07 1.141644e+07 611.0 64268502.0 3.314849e+09 15.560253 15275.0 5.660144e+09 0.000226 69.606270 138655.0 92.42 4.602690e+09 24003.0 1.881442e+08
In [16]:
print(type(dataset),"Data shape :",dataset.shape)
<class 'pandas.core.frame.DataFrame'> Data shape : (2174, 16)
In [17]:
dataset.shape
Out[17]:
(2174, 16)

Training Set.

In [18]:
len_day = 60
train_set = dataset.iloc[0:len(dataset)-len_day,:]
In [19]:
print(train_set.shape)
(2114, 16)
In [20]:
#save this data for the graph
X_train_4_graph = train_set['price(USD)']
In [21]:
X_train_4_graph= np.array(X_train_4_graph)

MinMax

In [22]:
from sklearn.preprocessing import MinMaxScaler
trainMinMax = MinMaxScaler()
train_set = trainMinMax.fit_transform(train_set)
In [23]:
train_set
Out[23]:
array([[1.59425211e-01, 1.07436574e-01, 9.94578482e-01, ...,
        6.31694873e-01, 1.08887965e-01, 4.27191948e-02],
       [1.54436671e-01, 3.58883943e-02, 1.00000000e+00, ...,
        6.30097832e-01, 1.02522496e-01, 1.69380282e-02],
       [1.05886211e-01, 3.46959389e-02, 1.00000000e+00, ...,
        6.29324742e-01, 9.76321707e-02, 1.60543006e-02],
       ...,
       [4.89532722e-03, 3.02233266e-04, 8.35279752e-07, ...,
        3.34344798e-05, 7.15668263e-03, 3.18461602e-04],
       [4.62234739e-03, 3.80901282e-04, 8.35279752e-07, ...,
        1.04332519e-05, 7.83167268e-03, 3.09342103e-04],
       [6.64972194e-03, 5.69653409e-04, 0.00000000e+00, ...,
        0.00000000e+00, 9.22635411e-03, 3.61063488e-04]])
In [24]:
y_train_4G= train_set[:,12]
y_train = y_train_4G.tolist()
In [25]:
#train_set=train_set.drop(train_set[:,12], axis=1)
#trainset =np.delete(train_set, train_set[:,12])
X_train_4Nor=train_set[:,(0,1,2,3,4,5,6,7,8,9,10,11,13,14,15)]
#X_train=train_set
X_train_3D = X_train_4Nor.tolist()
In [26]:
print (len(X_train_3D))
2114

<h1 style =color:red;"> Important change 2D to 3D array</h1>

-Samples. One sequence is one sample. A batch is comprised of one or more samples.

-Time Steps. One time step is one point of observation in the sample.

-Features. One feature is one observation at a time step.

In [27]:
#X_train = X_train.tolist
X_train_3D = np.reshape(X_train_3D, (2114, 15, 1))

Test Set

In [28]:
test_set = dataset.iloc[:60,:]

print (test_set.shape)
print (test_set.head(5))
(60, 16)
   activeAddresses  adjustedTxVolume(USD)  averageDifficulty  blockCount  \
0         102834.0           2.727544e+08       1.181510e+07       573.0   
1          99837.0           9.136417e+07       1.187950e+07       557.0   
2          70669.0           8.834103e+07       1.187950e+07       537.0   
3         140714.0           9.404545e+07       1.176827e+07       587.0   
4         162384.0           5.918720e+07       1.141644e+07       611.0   

     blockSize  exchangeVolume(USD)       fees  generatedCoins  \
0   65641701.0         3.461536e+09  17.753439         14325.0   
1   53092124.0         2.765901e+09  16.292335         13925.0   
2   14846398.0         2.742631e+09  16.982480         13425.0   
3  123559210.0         3.295696e+09  18.271835         14675.0   
4   64268502.0         3.314849e+09  15.560253         15275.0   

   marketcap(USD)  medianFee  medianTxValue(USD)  paymentCount  price(USD)  \
0    5.418036e+09   0.000221           75.837923       75246.0       88.39   
1    5.317491e+09   0.000222           66.125885       76006.0       86.77   
2    5.483492e+09   0.000222           80.628978       46219.0       89.50   
3    5.656034e+09   0.000226           78.122568       50448.0       92.33   
4    5.660144e+09   0.000226           69.606270      138655.0       92.42   

   realizedCap(USD)  txCount  txVolume(USD)  
0      4.632666e+09  26512.0   5.026659e+08  
1      4.621039e+09  25088.0   1.995761e+08  
2      4.615411e+09  23994.0   1.891868e+08  
3      4.607194e+09  25967.0   2.400780e+08  
4      4.602690e+09  24003.0   1.881442e+08  
In [29]:
y_test_4_graph = test_set['price(USD)']
y_test_4_graph= y_test_4_graph.reset_index(drop=True)
In [30]:
from sklearn.preprocessing import MinMaxScaler
testMinMax = MinMaxScaler()
test_set = testMinMax.fit_transform(test_set)#D
In [31]:
y_test = test_set[:,12]
y_test = y_test.tolist()
In [32]:
X_test_4N=test_set[:,(0,1,2,3,4,5,6,7,8,9,10,11,13,14,15)]
#X_train=train_set

#X_train = X_train.values
#X_train = sc.fit_transform(X_train)
X_test_3D = X_test_4N.tolist()
In [33]:
print (len(X_test_3D))
60
In [34]:
X_test_3D = np.reshape(X_test_3D, (60, 15, 1))
In [35]:
print (y_test)
[0.9206380464749901, 0.8887357227254824, 0.9424970460811344, 0.998227648680583, 1.0, 0.9269397400551398, 0.8546671918077986, 0.8572272548247342, 0.681764474202442, 0.37534462386766454, 0.37672311933832225, 0.37416305632138647, 0.3861756597085466, 0.38460023631350926, 0.4052776683733754, 0.3499409216226861, 0.35191020086648295, 0.3674675068924773, 0.385978731784167, 0.3574241827491138, 0.3477747144545096, 0.3786923985821189, 0.37495076801890503, 0.36589208349743996, 0.38676644348168576, 0.3985821189444665, 0.3416699487987398, 0.2934226073257187, 0.2800315084679008, 0.30405671524222133, 0.27116975187081516, 0.30622292241039784, 0.32414336352894835, 0.28278849940921624, 0.31114612051988977, 0.27963765261914153, 0.22410397794407255, 0.09610082709728252, 0.1317447814100039, 0.14533280819220173, 0.11500590783773135, 0.09058684521465143, 0.07837731390311142, 0.07601417881055539, 0.08940527766837336, 0.05789680976762501, 0.19771563607719567, 0.15813312327688067, 0.14749901536037813, 0.2008664828672706, 0.12170145726664039, 0.12800315084679015, 0.044111855061047756, 0.03465931469082317, 0.022055927530523767, 0.0, 0.004726270185112291, 0.043127215439149214, 0.030129972430090635, 0.10319023237495084]

Loss function

Mean Squared Error formula

LSTM is made up of three gates:

Forget Gate (f_t) - Controls if/when the context is forgotten. (MC)

Input Gate (i_t) - Controls if/when a value should be remembered by the context. (M+/MS)

Output Gate (o_t) - Controls if/when the remembered value is allowed to pass from the unit. (RM)

Sigmoid function.

Loss functions

Mean absolute error

mean absolute percentage error

Mean squared logarithmic error

Squared_hinge

Hinge

categorical hinge

logcosh

Categorical crossentropy,Spare categorical crossentorpy.

.....etc

Mean Squared Error formula

Gradient descent optimization algorithms

Stochastic gradient descent optimizer

RMSProp optimizer

Adagrad optimizer

Adadelta opimizer

Adamax, and Nadam

Adaptive Moment Estimation (Adam)

Two layer with Sigmoid and RELU activation fuction

In [36]:
sigRulu = Sequential()

#Adding the input layer and the LSTM layer
sigRulu.add(LSTM(units = 50, activation = 'sigmoid',return_sequences=True, input_shape = (None, 1)))
#regressor.add(LSTM(units=20,return_sequences=True))
#Adding the output layer
sigRulu.add(LSTM(units = 100, activation= 'relu', return_sequences=False))
sigRulu.add(Dropout(rate=.2))
sigRulu.add(Dense(units = 1))
#Compiling the Recurrent Neural Network
sigRulu.compile(optimizer = 'adam', loss = 'mean_squared_error')
WARNING:tensorflow:From C:\Users\gustl\Anaconda3\envs\Deeplearning\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From C:\Users\gustl\Anaconda3\envs\Deeplearning\lib\site-packages\keras\backend\tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.

In [37]:
historyTwo= sigRulu.fit(X_train_3D, y_train, batch_size = 50, epochs = 500)
WARNING:tensorflow:From C:\Users\gustl\Anaconda3\envs\Deeplearning\lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Epoch 1/500
2114/2114 [==============================] - 5s 2ms/step - loss: 0.0273
Epoch 2/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0225
Epoch 3/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0221
Epoch 4/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0215A: 0s - loss: 0.02
Epoch 5/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0213
Epoch 6/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0203
Epoch 7/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0191
Epoch 8/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0125
Epoch 9/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0103
Epoch 10/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0072
Epoch 11/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0071A: 0s - l
Epoch 12/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0061A: 0s - l
Epoch 13/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0054
Epoch 14/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0052
Epoch 15/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0050
Epoch 16/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0039A: 0s - loss
Epoch 17/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0062
Epoch 18/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0049
Epoch 19/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0039
Epoch 20/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0038
Epoch 21/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0038
Epoch 22/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0042
Epoch 23/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0038
Epoch 24/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0035
Epoch 25/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0038
Epoch 26/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0030A: 0s - loss: 0.00
Epoch 27/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0033
Epoch 28/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0031
Epoch 29/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0026
Epoch 30/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0027
Epoch 31/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0030
Epoch 32/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0027
Epoch 33/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0024
Epoch 34/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0027
Epoch 35/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0034
Epoch 36/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0029A: 0s - los
Epoch 37/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0023
Epoch 38/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0024
Epoch 39/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0028
Epoch 40/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0026
Epoch 41/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0025
Epoch 42/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0024
Epoch 43/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0023
Epoch 44/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0026
Epoch 45/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0020
Epoch 46/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0019
Epoch 47/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0021
Epoch 48/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0023
Epoch 49/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0021
Epoch 50/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0021
Epoch 51/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0020
Epoch 52/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0018
Epoch 53/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0018
Epoch 54/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0021A: 0s - 
Epoch 55/500
2114/2114 [==============================] - 5s 2ms/step - loss: 0.0019
Epoch 56/500
2114/2114 [==============================] - 8s 4ms/step - loss: 0.0022
Epoch 57/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0019
Epoch 58/500
2114/2114 [==============================] - 5s 2ms/step - loss: 0.0020
Epoch 59/500
2114/2114 [==============================] - 7s 3ms/step - loss: 0.0021
Epoch 60/500
2114/2114 [==============================] - 5s 2ms/step - loss: 0.0018
Epoch 61/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0016
Epoch 62/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0019
Epoch 63/500
2114/2114 [==============================] - 2s 750us/step - loss: 0.0018
Epoch 64/500
2114/2114 [==============================] - 1s 702us/step - loss: 0.0017
Epoch 65/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0018
Epoch 66/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0014
Epoch 67/500
2114/2114 [==============================] - 2s 943us/step - loss: 0.0016
Epoch 68/500
2114/2114 [==============================] - 2s 891us/step - loss: 0.0015
Epoch 69/500
2114/2114 [==============================] - 2s 894us/step - loss: 0.0016
Epoch 70/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0018
Epoch 71/500
2114/2114 [==============================] - 1s 643us/step - loss: 0.0018
Epoch 72/500
2114/2114 [==============================] - 1s 699us/step - loss: 0.0016
Epoch 73/500
2114/2114 [==============================] - 1s 645us/step - loss: 0.0016
Epoch 74/500
2114/2114 [==============================] - 1s 703us/step - loss: 0.0014
Epoch 75/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0016
Epoch 76/500
2114/2114 [==============================] - 2s 883us/step - loss: 0.0014
Epoch 77/500
2114/2114 [==============================] - 2s 874us/step - loss: 0.0014
Epoch 78/500
2114/2114 [==============================] - 2s 849us/step - loss: 0.0018
Epoch 79/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0014
Epoch 80/500
2114/2114 [==============================] - 2s 919us/step - loss: 0.0015
Epoch 81/500
2114/2114 [==============================] - 2s 940us/step - loss: 0.0013
Epoch 82/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0016
Epoch 83/500
2114/2114 [==============================] - 2s 996us/step - loss: 0.0014
Epoch 84/500
2114/2114 [==============================] - 2s 893us/step - loss: 0.0012
Epoch 85/500
2114/2114 [==============================] - 2s 897us/step - loss: 0.0014
Epoch 86/500
2114/2114 [==============================] - 2s 864us/step - loss: 0.0014
Epoch 87/500
2114/2114 [==============================] - 2s 848us/step - loss: 0.0014
Epoch 88/500
2114/2114 [==============================] - 2s 937us/step - loss: 0.0014
Epoch 89/500
2114/2114 [==============================] - 2s 852us/step - loss: 0.0014
Epoch 90/500
2114/2114 [==============================] - 2s 762us/step - loss: 0.0017
Epoch 91/500
2114/2114 [==============================] - 1s 623us/step - loss: 0.0014
Epoch 92/500
2114/2114 [==============================] - 1s 668us/step - loss: 0.0011 0s - loss: 0
Epoch 93/500
2114/2114 [==============================] - 1s 598us/step - loss: 0.0012 0s -
Epoch 94/500
2114/2114 [==============================] - 1s 640us/step - loss: 0.0013
Epoch 95/500
2114/2114 [==============================] - 1s 656us/step - loss: 0.0013 0s - l
Epoch 96/500
2114/2114 [==============================] - 2s 930us/step - loss: 0.0014
Epoch 97/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0013
Epoch 98/500
2114/2114 [==============================] - 2s 726us/step - loss: 0.0014
Epoch 99/500
2114/2114 [==============================] - 2s 855us/step - loss: 0.0014
Epoch 100/500
2114/2114 [==============================] - 2s 835us/step - loss: 0.0013
Epoch 101/500
2114/2114 [==============================] - 2s 865us/step - loss: 0.0012
Epoch 102/500
2114/2114 [==============================] - 2s 860us/step - loss: 0.0014
Epoch 103/500
2114/2114 [==============================] - 2s 849us/step - loss: 0.0011
Epoch 104/500
2114/2114 [==============================] - 2s 844us/step - loss: 0.0015
Epoch 105/500
2114/2114 [==============================] - 2s 866us/step - loss: 0.0015
Epoch 106/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0012
Epoch 107/500
2114/2114 [==============================] - 2s 867us/step - loss: 0.0015
Epoch 108/500
2114/2114 [==============================] - 2s 878us/step - loss: 0.0012
Epoch 109/500
2114/2114 [==============================] - 2s 901us/step - loss: 0.0012
Epoch 110/500
2114/2114 [==============================] - 2s 835us/step - loss: 0.0012
Epoch 111/500
2114/2114 [==============================] - 1s 605us/step - loss: 0.0014
Epoch 112/500
2114/2114 [==============================] - 1s 684us/step - loss: 0.0012
Epoch 113/500
2114/2114 [==============================] - 2s 896us/step - loss: 9.9960e-04
Epoch 114/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0011- ETA
Epoch 115/500
2114/2114 [==============================] - 2s 1ms/step - loss: 9.9821e-04
Epoch 116/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0012
Epoch 117/500
2114/2114 [==============================] - 2s 952us/step - loss: 0.0012
Epoch 118/500
2114/2114 [==============================] - 1s 703us/step - loss: 0.0015
Epoch 119/500
2114/2114 [==============================] - 1s 635us/step - loss: 0.0012
Epoch 120/500
2114/2114 [==============================] - 1s 612us/step - loss: 0.0010
Epoch 121/500
2114/2114 [==============================] - 1s 609us/step - loss: 0.0012
Epoch 122/500
2114/2114 [==============================] - 1s 680us/step - loss: 0.0010
Epoch 123/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0010A: 0s - loss: 0
Epoch 124/500
2114/2114 [==============================] - 2s 940us/step - loss: 0.0011
Epoch 125/500
2114/2114 [==============================] - 2s 821us/step - loss: 0.0012
Epoch 126/500
2114/2114 [==============================] - 2s 721us/step - loss: 0.0015
Epoch 127/500
2114/2114 [==============================] - 2s 747us/step - loss: 0.0011
Epoch 128/500
2114/2114 [==============================] - 2s 747us/step - loss: 0.0011
Epoch 129/500
2114/2114 [==============================] - 2s 749us/step - loss: 0.0011
Epoch 130/500
2114/2114 [==============================] - 2s 717us/step - loss: 0.0010
Epoch 131/500
2114/2114 [==============================] - 2s 752us/step - loss: 0.0011
Epoch 132/500
2114/2114 [==============================] - 2s 722us/step - loss: 8.9259e-04
Epoch 133/500
2114/2114 [==============================] - 2s 909us/step - loss: 0.0010
Epoch 134/500
2114/2114 [==============================] - 1s 657us/step - loss: 0.0012
Epoch 135/500
2114/2114 [==============================] - 2s 746us/step - loss: 0.0010
Epoch 136/500
2114/2114 [==============================] - 2s 729us/step - loss: 9.0086e-04
Epoch 137/500
2114/2114 [==============================] - 2s 764us/step - loss: 0.0011
Epoch 138/500
2114/2114 [==============================] - 1s 677us/step - loss: 0.0011
Epoch 139/500
2114/2114 [==============================] - 1s 628us/step - loss: 0.0011
Epoch 140/500
2114/2114 [==============================] - 1s 620us/step - loss: 8.0639e-04
Epoch 141/500
2114/2114 [==============================] - 1s 647us/step - loss: 0.0011
Epoch 142/500
2114/2114 [==============================] - 1s 615us/step - loss: 0.0011 0s - l
Epoch 143/500
2114/2114 [==============================] - 1s 612us/step - loss: 0.0011
Epoch 144/500
2114/2114 [==============================] - 2s 942us/step - loss: 0.0012
Epoch 145/500
2114/2114 [==============================] - 2s 777us/step - loss: 0.0010
Epoch 146/500
2114/2114 [==============================] - 2s 742us/step - loss: 9.3139e-04
Epoch 147/500
2114/2114 [==============================] - 2s 774us/step - loss: 8.2794e-04
Epoch 148/500
2114/2114 [==============================] - 2s 728us/step - loss: 9.8824e-04
Epoch 149/500
2114/2114 [==============================] - 1s 708us/step - loss: 0.0013
Epoch 150/500
2114/2114 [==============================] - 1s 685us/step - loss: 8.9507e-04
Epoch 151/500
2114/2114 [==============================] - 1s 709us/step - loss: 8.7181e-04
Epoch 152/500
2114/2114 [==============================] - 2s 760us/step - loss: 0.0010
Epoch 153/500
2114/2114 [==============================] - 2s 721us/step - loss: 0.0011
Epoch 154/500
2114/2114 [==============================] - 2s 981us/step - loss: 0.0012
Epoch 155/500
2114/2114 [==============================] - 2s 753us/step - loss: 0.0012
Epoch 156/500
2114/2114 [==============================] - 2s 797us/step - loss: 9.2299e-04
Epoch 157/500
2114/2114 [==============================] - 2s 729us/step - loss: 9.7340e-04
Epoch 158/500
2114/2114 [==============================] - 1s 682us/step - loss: 9.7399e-04
Epoch 159/500
2114/2114 [==============================] - 1s 704us/step - loss: 8.2802e-04
Epoch 160/500
2114/2114 [==============================] - 1s 681us/step - loss: 8.1916e-04
Epoch 161/500
2114/2114 [==============================] - 1s 680us/step - loss: 0.0011
Epoch 162/500
2114/2114 [==============================] - 1s 626us/step - loss: 0.0010
Epoch 163/500
2114/2114 [==============================] - 1s 615us/step - loss: 9.4580e-04
Epoch 164/500
2114/2114 [==============================] - 2s 789us/step - loss: 9.9404e-04
Epoch 165/500
2114/2114 [==============================] - 1s 641us/step - loss: 7.3200e-04
Epoch 166/500
2114/2114 [==============================] - 1s 626us/step - loss: 0.0011
Epoch 167/500
2114/2114 [==============================] - 1s 623us/step - loss: 0.0010
Epoch 168/500
2114/2114 [==============================] - 1s 640us/step - loss: 9.0563e-04
Epoch 169/500
2114/2114 [==============================] - 1s 643us/step - loss: 7.6261e-04
Epoch 170/500
2114/2114 [==============================] - 1s 617us/step - loss: 9.8892e-04
Epoch 171/500
2114/2114 [==============================] - 1s 667us/step - loss: 0.0012
Epoch 172/500
2114/2114 [==============================] - 2s 715us/step - loss: 7.8527e-04
Epoch 173/500
2114/2114 [==============================] - 2s 771us/step - loss: 8.8936e-04
Epoch 174/500
2114/2114 [==============================] - 2s 960us/step - loss: 8.1204e-04
Epoch 175/500
2114/2114 [==============================] - 2s 984us/step - loss: 7.3187e-04
Epoch 176/500
2114/2114 [==============================] - 2s 781us/step - loss: 8.4684e-04
Epoch 177/500
2114/2114 [==============================] - 2s 733us/step - loss: 9.5787e-04
Epoch 178/500
2114/2114 [==============================] - ETA: 0s - loss: 8.6552e-0 - 2s 777us/step - loss: 8.6162e-04
Epoch 179/500
2114/2114 [==============================] - 2s 924us/step - loss: 7.4668e-04
Epoch 180/500
2114/2114 [==============================] - 2s 745us/step - loss: 9.3215e-04
Epoch 181/500
2114/2114 [==============================] - 1s 699us/step - loss: 8.7018e-04
Epoch 182/500
2114/2114 [==============================] - 1s 675us/step - loss: 7.9231e-04
Epoch 183/500
2114/2114 [==============================] - 1s 608us/step - loss: 9.6376e-04
Epoch 184/500
2114/2114 [==============================] - 1s 599us/step - loss: 0.0012
Epoch 185/500
2114/2114 [==============================] - 2s 892us/step - loss: 0.0010 0s - loss:
Epoch 186/500
2114/2114 [==============================] - 2s 740us/step - loss: 7.7623e-04
Epoch 187/500
2114/2114 [==============================] - 2s 747us/step - loss: 6.5012e-04
Epoch 188/500
2114/2114 [==============================] - 2s 750us/step - loss: 9.1309e-04
Epoch 189/500
2114/2114 [==============================] - 2s 778us/step - loss: 0.0011
Epoch 190/500
2114/2114 [==============================] - 2s 744us/step - loss: 8.8935e-04
Epoch 191/500
2114/2114 [==============================] - 2s 774us/step - loss: 7.4841e-04
Epoch 192/500
2114/2114 [==============================] - 2s 731us/step - loss: 8.2450e-04
Epoch 193/500
2114/2114 [==============================] - 2s 756us/step - loss: 7.6809e-04 
Epoch 194/500
2114/2114 [==============================] - 2s 750us/step - loss: 9.7036e-04
Epoch 195/500
2114/2114 [==============================] - 2s 955us/step - loss: 7.6339e-04
Epoch 196/500
2114/2114 [==============================] - 2s 779us/step - loss: 8.6418e-04
Epoch 197/500
2114/2114 [==============================] - 2s 748us/step - loss: 7.0891e-04
Epoch 198/500
2114/2114 [==============================] - 2s 782us/step - loss: 8.3600e-04
Epoch 199/500
2114/2114 [==============================] - 2s 735us/step - loss: 8.6136e-04
Epoch 200/500
2114/2114 [==============================] - 2s 761us/step - loss: 8.2144e-04
Epoch 201/500
2114/2114 [==============================] - 2s 895us/step - loss: 7.0822e-04
Epoch 202/500
2114/2114 [==============================] - 2s 752us/step - loss: 7.8139e-04
Epoch 203/500
2114/2114 [==============================] - 1s 646us/step - loss: 7.7632e-04
Epoch 204/500
2114/2114 [==============================] - 2s 741us/step - loss: 9.4438e-04
Epoch 205/500
2114/2114 [==============================] - 2s 862us/step - loss: 7.4195e-04
Epoch 206/500
2114/2114 [==============================] - 1s 636us/step - loss: 7.4429e-04
Epoch 207/500
2114/2114 [==============================] - 1s 641us/step - loss: 7.4546e-04
Epoch 208/500
2114/2114 [==============================] - 1s 672us/step - loss: 8.2858e-04
Epoch 209/500
2114/2114 [==============================] - 2s 771us/step - loss: 6.6167e-04
Epoch 210/500
2114/2114 [==============================] - 2s 750us/step - loss: 7.6839e-04
Epoch 211/500
2114/2114 [==============================] - 2s 743us/step - loss: 7.5645e-04
Epoch 212/500
2114/2114 [==============================] - 2s 724us/step - loss: 7.3018e-04
Epoch 213/500
2114/2114 [==============================] - 2s 752us/step - loss: 8.6711e-04
Epoch 214/500
2114/2114 [==============================] - 2s 866us/step - loss: 7.2123e-04
Epoch 215/500
2114/2114 [==============================] - 2s 1ms/step - loss: 6.8400e-04
Epoch 216/500
2114/2114 [==============================] - 2s 764us/step - loss: 6.7893e-04
Epoch 217/500
2114/2114 [==============================] - 2s 748us/step - loss: 7.2194e-04
Epoch 218/500
2114/2114 [==============================] - 2s 752us/step - loss: 7.6359e-04
Epoch 219/500
2114/2114 [==============================] - 2s 739us/step - loss: 8.0602e-04
Epoch 220/500
2114/2114 [==============================] - 2s 718us/step - loss: 7.1464e-04
Epoch 221/500
2114/2114 [==============================] - 1s 639us/step - loss: 7.6278e-04
Epoch 222/500
2114/2114 [==============================] - 1s 603us/step - loss: 8.3673e-04
Epoch 223/500
2114/2114 [==============================] - 1s 634us/step - loss: 7.0911e-04
Epoch 224/500
2114/2114 [==============================] - 1s 610us/step - loss: 6.3766e-04
Epoch 225/500
2114/2114 [==============================] - 2s 789us/step - loss: 7.4521e-04
Epoch 226/500
2114/2114 [==============================] - 1s 670us/step - loss: 6.6412e-04
Epoch 227/500
2114/2114 [==============================] - 1s 620us/step - loss: 7.9401e-04 1s 
Epoch 228/500
2114/2114 [==============================] - 1s 642us/step - loss: 7.0297e-04
Epoch 229/500
2114/2114 [==============================] - 1s 626us/step - loss: 7.1951e-04
Epoch 230/500
2114/2114 [==============================] - 1s 652us/step - loss: 0.0011
Epoch 231/500
2114/2114 [==============================] - 1s 671us/step - loss: 6.9370e-04
Epoch 232/500
2114/2114 [==============================] - 2s 722us/step - loss: 6.3560e-04
Epoch 233/500
2114/2114 [==============================] - 2s 784us/step - loss: 7.1803e-04
Epoch 234/500
2114/2114 [==============================] - 2s 751us/step - loss: 6.6786e-04
Epoch 235/500
2114/2114 [==============================] - 2s 775us/step - loss: 7.4447e-04
Epoch 236/500
2114/2114 [==============================] - 2s 1ms/step - loss: 7.0441e-04
Epoch 237/500
2114/2114 [==============================] - 2s 989us/step - loss: 7.0682e-04
Epoch 238/500
2114/2114 [==============================] - 2s 854us/step - loss: 5.7133e-04
Epoch 239/500
2114/2114 [==============================] - 6s 3ms/step - loss: 6.1675e-04
Epoch 240/500
2114/2114 [==============================] - 7s 3ms/step - loss: 7.7751e-04
Epoch 241/500
2114/2114 [==============================] - 4s 2ms/step - loss: 7.3284e-04
Epoch 242/500
2114/2114 [==============================] - 3s 2ms/step - loss: 8.2170e-04
Epoch 243/500
2114/2114 [==============================] - 3s 2ms/step - loss: 5.4496e-04
Epoch 244/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.9381e-04
Epoch 245/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.6142e-04
Epoch 246/500
2114/2114 [==============================] - 4s 2ms/step - loss: 7.4846e-04
Epoch 247/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.2210e-04
Epoch 248/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.7503e-04
Epoch 249/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.2241e-04
Epoch 250/500
2114/2114 [==============================] - 5s 2ms/step - loss: 6.6197e-04
Epoch 251/500
2114/2114 [==============================] - 3s 1ms/step - loss: 7.2795e-04
Epoch 252/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.9452e-04
Epoch 253/500
2114/2114 [==============================] - 3s 1ms/step - loss: 7.1044e-04
Epoch 254/500
2114/2114 [==============================] - 2s 822us/step - loss: 6.8541e-04
Epoch 255/500
2114/2114 [==============================] - 2s 756us/step - loss: 7.6790e-04
Epoch 256/500
2114/2114 [==============================] - 2s 796us/step - loss: 7.6198e-04
Epoch 257/500
2114/2114 [==============================] - 2s 811us/step - loss: 7.4339e-04
Epoch 258/500
2114/2114 [==============================] - 2s 773us/step - loss: 7.6611e-04
Epoch 259/500
2114/2114 [==============================] - 2s 729us/step - loss: 6.5987e-04
Epoch 260/500
2114/2114 [==============================] - 2s 911us/step - loss: 7.5603e-04
Epoch 261/500
2114/2114 [==============================] - 2s 819us/step - loss: 6.4439e-04
Epoch 262/500
2114/2114 [==============================] - 2s 780us/step - loss: 6.3695e-04
Epoch 263/500
2114/2114 [==============================] - 2s 764us/step - loss: 8.1259e-04
Epoch 264/500
2114/2114 [==============================] - 2s 725us/step - loss: 7.9445e-04
Epoch 265/500
2114/2114 [==============================] - 2s 790us/step - loss: 6.8279e-04
Epoch 266/500
2114/2114 [==============================] - 2s 920us/step - loss: 5.7711e-04
Epoch 267/500
2114/2114 [==============================] - 2s 757us/step - loss: 6.9081e-04
Epoch 268/500
2114/2114 [==============================] - 2s 719us/step - loss: 7.3989e-04
Epoch 269/500
2114/2114 [==============================] - 1s 658us/step - loss: 7.2062e-04
Epoch 270/500
2114/2114 [==============================] - 2s 862us/step - loss: 6.6513e-04
Epoch 271/500
2114/2114 [==============================] - 2s 786us/step - loss: 6.5107e-04
Epoch 272/500
2114/2114 [==============================] - 2s 908us/step - loss: 7.1964e-04
Epoch 273/500
2114/2114 [==============================] - 2s 1ms/step - loss: 7.1027e-04
Epoch 274/500
2114/2114 [==============================] - 2s 881us/step - loss: 5.8926e-04
Epoch 275/500
2114/2114 [==============================] - 3s 1ms/step - loss: 6.7295e-04
Epoch 276/500
2114/2114 [==============================] - 2s 945us/step - loss: 7.5125e-04
Epoch 277/500
2114/2114 [==============================] - 1s 631us/step - loss: 6.1076e-04 0s - lo
Epoch 278/500
2114/2114 [==============================] - 1s 610us/step - loss: 7.5038e-04
Epoch 279/500
2114/2114 [==============================] - 2s 837us/step - loss: 6.2206e-04 0s - loss: 6.331
Epoch 280/500
2114/2114 [==============================] - 1s 624us/step - loss: 5.3866e-04
Epoch 281/500
2114/2114 [==============================] - 1s 618us/step - loss: 5.0907e-04
Epoch 282/500
2114/2114 [==============================] - 1s 638us/step - loss: 6.8338e-04
Epoch 283/500
2114/2114 [==============================] - 1s 603us/step - loss: 5.9045e-04
Epoch 284/500
2114/2114 [==============================] - 2s 723us/step - loss: 5.3552e-04
Epoch 285/500
2114/2114 [==============================] - 1s 550us/step - loss: 6.1503e-04
Epoch 286/500
2114/2114 [==============================] - 1s 591us/step - loss: 5.2869e-04
Epoch 287/500
2114/2114 [==============================] - 1s 600us/step - loss: 6.3775e-04
Epoch 288/500
2114/2114 [==============================] - 1s 666us/step - loss: 5.7331e-04
Epoch 289/500
2114/2114 [==============================] - 3s 1ms/step - loss: 6.3368e-04
Epoch 290/500
2114/2114 [==============================] - 2s 756us/step - loss: 6.5094e-04
Epoch 291/500
2114/2114 [==============================] - 3s 1ms/step - loss: 6.8767e-04
Epoch 292/500
2114/2114 [==============================] - 2s 1ms/step - loss: 6.6660e-04
Epoch 293/500
2114/2114 [==============================] - 3s 1ms/step - loss: 6.5010e-04
Epoch 294/500
2114/2114 [==============================] - 2s 1ms/step - loss: 5.3555e-04
Epoch 295/500
2114/2114 [==============================] - 2s 858us/step - loss: 6.5846e-04
Epoch 296/500
2114/2114 [==============================] - 2s 720us/step - loss: 6.2281e-04
Epoch 297/500
2114/2114 [==============================] - 2s 768us/step - loss: 5.8899e-04
Epoch 298/500
2114/2114 [==============================] - 1s 632us/step - loss: 6.2133e-04
Epoch 299/500
2114/2114 [==============================] - 1s 612us/step - loss: 6.1503e-04
Epoch 300/500
2114/2114 [==============================] - 1s 643us/step - loss: 5.8990e-04
Epoch 301/500
2114/2114 [==============================] - 1s 634us/step - loss: 5.7272e-04
Epoch 302/500
2114/2114 [==============================] - 2s 793us/step - loss: 5.7459e-04 0s -
Epoch 303/500
2114/2114 [==============================] - 2s 1ms/step - loss: 7.2124e-04
Epoch 304/500
2114/2114 [==============================] - 2s 753us/step - loss: 6.4217e-04
Epoch 305/500
2114/2114 [==============================] - 2s 725us/step - loss: 6.7751e-04
Epoch 306/500
2114/2114 [==============================] - 2s 712us/step - loss: 6.0031e-04
Epoch 307/500
2114/2114 [==============================] - 2s 784us/step - loss: 6.0570e-04
Epoch 308/500
2114/2114 [==============================] - 1s 656us/step - loss: 6.3476e-04
Epoch 309/500
2114/2114 [==============================] - 1s 618us/step - loss: 6.3617e-04 0s - loss: 6.
Epoch 310/500
2114/2114 [==============================] - 1s 643us/step - loss: 5.7726e-04
Epoch 311/500
2114/2114 [==============================] - 1s 606us/step - loss: 6.0358e-04 0s - loss: 6.0758e-0
Epoch 312/500
2114/2114 [==============================] - 1s 627us/step - loss: 6.2932e-04
Epoch 313/500
2114/2114 [==============================] - 1s 622us/step - loss: 5.5504e-04
Epoch 314/500
2114/2114 [==============================] - 1s 652us/step - loss: 7.3032e-04
Epoch 315/500
2114/2114 [==============================] - 1s 681us/step - loss: 6.0508e-04
Epoch 316/500
2114/2114 [==============================] - 1s 656us/step - loss: 5.8187e-04
Epoch 317/500
2114/2114 [==============================] - 1s 615us/step - loss: 6.9199e-04
Epoch 318/500
2114/2114 [==============================] - 1s 664us/step - loss: 5.9547e-04
Epoch 319/500
2114/2114 [==============================] - 2s 869us/step - loss: 5.7460e-04
Epoch 320/500
2114/2114 [==============================] - 1s 677us/step - loss: 6.8495e-04
Epoch 321/500
2114/2114 [==============================] - 1s 654us/step - loss: 7.1106e-04 0s - loss: 
Epoch 322/500
2114/2114 [==============================] - 2s 732us/step - loss: 6.1283e-04 0s - loss: 
Epoch 323/500
2114/2114 [==============================] - 1s 649us/step - loss: 5.7835e-04
Epoch 324/500
2114/2114 [==============================] - 1s 682us/step - loss: 5.1961e-04
Epoch 325/500
2114/2114 [==============================] - 1s 614us/step - loss: 6.0748e-04 0s - loss: 5
Epoch 326/500
2114/2114 [==============================] - 2s 757us/step - loss: 6.9119e-04
Epoch 327/500
2114/2114 [==============================] - 1s 599us/step - loss: 7.5606e-04
Epoch 328/500
2114/2114 [==============================] - 1s 604us/step - loss: 6.3497e-04 1s 
Epoch 329/500
2114/2114 [==============================] - 1s 612us/step - loss: 6.3854e-04
Epoch 330/500
2114/2114 [==============================] - 2s 845us/step - loss: 5.7380e-04
Epoch 331/500
2114/2114 [==============================] - 1s 634us/step - loss: 5.7344e-04
Epoch 332/500
2114/2114 [==============================] - 1s 619us/step - loss: 6.3637e-04
Epoch 333/500
2114/2114 [==============================] - 1s 612us/step - loss: 5.8521e-04 0s - loss: 6.8 - ETA: 0s - loss: 
Epoch 334/500
2114/2114 [==============================] - 1s 614us/step - loss: 5.5144e-04
Epoch 335/500
2114/2114 [==============================] - 1s 610us/step - loss: 5.4620e-04
Epoch 336/500
2114/2114 [==============================] - 1s 626us/step - loss: 6.7086e-04 1s - 
Epoch 337/500
2114/2114 [==============================] - 1s 633us/step - loss: 5.4323e-04 0s - loss: 5.4138e
Epoch 338/500
2114/2114 [==============================] - 1s 610us/step - loss: 6.1880e-04
Epoch 339/500
2114/2114 [==============================] - 1s 629us/step - loss: 6.4150e-04 0s - loss
Epoch 340/500
2114/2114 [==============================] - 1s 624us/step - loss: 6.7502e-04
Epoch 341/500
2114/2114 [==============================] - 1s 603us/step - loss: 5.4461e-04
Epoch 342/500
2114/2114 [==============================] - 2s 801us/step - loss: 6.0410e-04
Epoch 343/500
2114/2114 [==============================] - 1s 627us/step - loss: 7.5353e-04
Epoch 344/500
2114/2114 [==============================] - 1s 617us/step - loss: 7.1398e-04
Epoch 345/500
2114/2114 [==============================] - 1s 619us/step - loss: 5.8870e-04
Epoch 346/500
2114/2114 [==============================] - 1s 613us/step - loss: 5.4426e-04 0s - los
Epoch 347/500
2114/2114 [==============================] - 1s 622us/step - loss: 6.5810e-04 0s - loss: 
Epoch 348/500
2114/2114 [==============================] - 1s 622us/step - loss: 6.9828e-04
Epoch 349/500
2114/2114 [==============================] - 1s 621us/step - loss: 6.0089e-04
Epoch 350/500
2114/2114 [==============================] - 1s 607us/step - loss: 6.8061e-04
Epoch 351/500
2114/2114 [==============================] - 1s 627us/step - loss: 5.8606e-04
Epoch 352/500
2114/2114 [==============================] - 1s 605us/step - loss: 6.2388e-04
Epoch 353/500
2114/2114 [==============================] - 1s 611us/step - loss: 5.5643e-04
Epoch 354/500
2114/2114 [==============================] - 2s 819us/step - loss: 5.9960e-04
Epoch 355/500
2114/2114 [==============================] - 1s 621us/step - loss: 6.4559e-04
Epoch 356/500
2114/2114 [==============================] - 1s 610us/step - loss: 6.5849e-04
Epoch 357/500
2114/2114 [==============================] - 1s 619us/step - loss: 5.0255e-04
Epoch 358/500
2114/2114 [==============================] - 1s 623us/step - loss: 5.7631e-04
Epoch 359/500
2114/2114 [==============================] - 1s 600us/step - loss: 5.9615e-04
Epoch 360/500
2114/2114 [==============================] - 1s 640us/step - loss: 7.3339e-04
Epoch 361/500
2114/2114 [==============================] - 1s 621us/step - loss: 8.0141e-04 0s 
Epoch 362/500
2114/2114 [==============================] - 1s 617us/step - loss: 7.3997e-04
Epoch 363/500
2114/2114 [==============================] - 1s 589us/step - loss: 7.1684e-04
Epoch 364/500
2114/2114 [==============================] - 1s 598us/step - loss: 8.1935e-04
Epoch 365/500
2114/2114 [==============================] - 1s 593us/step - loss: 6.8246e-04
Epoch 366/500
2114/2114 [==============================] - 2s 781us/step - loss: 5.4010e-04
Epoch 367/500
2114/2114 [==============================] - 1s 610us/step - loss: 6.4050e-04
Epoch 368/500
2114/2114 [==============================] - 1s 601us/step - loss: 6.6346e-04
Epoch 369/500
2114/2114 [==============================] - 1s 605us/step - loss: 6.2649e-04
Epoch 370/500
2114/2114 [==============================] - 1s 591us/step - loss: 5.7022e-04
Epoch 371/500
2114/2114 [==============================] - 1s 600us/step - loss: 6.0294e-04 0s - loss: 5.894 - ETA: 0s - loss: 5.9180
Epoch 372/500
2114/2114 [==============================] - 1s 618us/step - loss: 6.7088e-04 0s - loss: 6 - ETA: 0s - loss: 6.6501e-
Epoch 373/500
2114/2114 [==============================] - 1s 595us/step - loss: 6.5280e-04
Epoch 374/500
2114/2114 [==============================] - 1s 612us/step - loss: 6.4485e-04
Epoch 375/500
2114/2114 [==============================] - 1s 615us/step - loss: 5.6764e-04
Epoch 376/500
2114/2114 [==============================] - 1s 603us/step - loss: 5.4628e-04 0s - loss: 5.6061e
Epoch 377/500
2114/2114 [==============================] - 1s 596us/step - loss: 5.7063e-04
Epoch 378/500
2114/2114 [==============================] - 2s 776us/step - loss: 5.1238e-04
Epoch 379/500
2114/2114 [==============================] - 1s 637us/step - loss: 5.7623e-04
Epoch 380/500
2114/2114 [==============================] - 1s 609us/step - loss: 6.6705e-04
Epoch 381/500
2114/2114 [==============================] - 1s 669us/step - loss: 6.3447e-04
Epoch 382/500
2114/2114 [==============================] - 2s 721us/step - loss: 6.4624e-04
Epoch 383/500
2114/2114 [==============================] - 2s 765us/step - loss: 6.4104e-04
Epoch 384/500
2114/2114 [==============================] - 2s 737us/step - loss: 6.1133e-04
Epoch 385/500
2114/2114 [==============================] - 2s 850us/step - loss: 5.6873e-04
Epoch 386/500
2114/2114 [==============================] - 1s 704us/step - loss: 5.2896e-04 0s - loss: 4.
Epoch 387/500
2114/2114 [==============================] - 2s 732us/step - loss: 5.4932e-04
Epoch 388/500
2114/2114 [==============================] - 2s 731us/step - loss: 5.8795e-04
Epoch 389/500
2114/2114 [==============================] - 2s 877us/step - loss: 5.8891e-04
Epoch 390/500
2114/2114 [==============================] - 1s 700us/step - loss: 5.8394e-04
Epoch 391/500
2114/2114 [==============================] - 1s 611us/step - loss: 5.5280e-04 0s - loss: 5.6317e-
Epoch 392/500
2114/2114 [==============================] - 1s 600us/step - loss: 5.2377e-04
Epoch 393/500
2114/2114 [==============================] - 1s 630us/step - loss: 6.8849e-04
Epoch 394/500
2114/2114 [==============================] - 1s 693us/step - loss: 6.9413e-04
Epoch 395/500
2114/2114 [==============================] - 1s 698us/step - loss: 6.1279e-04
Epoch 396/500
2114/2114 [==============================] - 1s 699us/step - loss: 6.9257e-04
Epoch 397/500
2114/2114 [==============================] - 1s 684us/step - loss: 4.3880e-04
Epoch 398/500
2114/2114 [==============================] - 1s 705us/step - loss: 6.0916e-04
Epoch 399/500
2114/2114 [==============================] - 2s 757us/step - loss: 5.5144e-04
Epoch 400/500
2114/2114 [==============================] - 2s 729us/step - loss: 5.3151e-04
Epoch 401/500
2114/2114 [==============================] - 1s 704us/step - loss: 5.5134e-04
Epoch 402/500
2114/2114 [==============================] - 1s 618us/step - loss: 5.7441e-04
Epoch 403/500
2114/2114 [==============================] - 1s 686us/step - loss: 5.6589e-04
Epoch 404/500
2114/2114 [==============================] - 2s 712us/step - loss: 5.3812e-04
Epoch 405/500
2114/2114 [==============================] - 1s 638us/step - loss: 6.5880e-04 0s - loss: 6.7209e-0 - ETA: 0s - loss: 6
Epoch 406/500
2114/2114 [==============================] - 1s 700us/step - loss: 5.5460e-04
Epoch 407/500
2114/2114 [==============================] - 1s 637us/step - loss: 5.7486e-04
Epoch 408/500
2114/2114 [==============================] - 1s 621us/step - loss: 5.2520e-04
Epoch 409/500
2114/2114 [==============================] - 1s 650us/step - loss: 5.1756e-04
Epoch 410/500
2114/2114 [==============================] - 1s 651us/step - loss: 6.0069e-04
Epoch 411/500
2114/2114 [==============================] - 2s 828us/step - loss: 5.8862e-04
Epoch 412/500
2114/2114 [==============================] - 2s 787us/step - loss: 6.8862e-04
Epoch 413/500
2114/2114 [==============================] - 1s 604us/step - loss: 5.9450e-04
Epoch 414/500
2114/2114 [==============================] - 1s 600us/step - loss: 7.0046e-04
Epoch 415/500
2114/2114 [==============================] - 1s 633us/step - loss: 4.8695e-04
Epoch 416/500
2114/2114 [==============================] - 1s 652us/step - loss: 5.5362e-04
Epoch 417/500
2114/2114 [==============================] - 1s 617us/step - loss: 5.7044e-04
Epoch 418/500
2114/2114 [==============================] - 1s 607us/step - loss: 6.5182e-04
Epoch 419/500
2114/2114 [==============================] - 1s 632us/step - loss: 7.1265e-04
Epoch 420/500
2114/2114 [==============================] - 1s 612us/step - loss: 5.1906e-04 0s - loss: 6.
Epoch 421/500
2114/2114 [==============================] - 1s 664us/step - loss: 5.1246e-04
Epoch 422/500
2114/2114 [==============================] - 2s 932us/step - loss: 6.3971e-04
Epoch 423/500
2114/2114 [==============================] - 1s 679us/step - loss: 5.5048e-04
Epoch 424/500
2114/2114 [==============================] - 1s 681us/step - loss: 5.3216e-04
Epoch 425/500
2114/2114 [==============================] - 1s 680us/step - loss: 5.5689e-04
Epoch 426/500
2114/2114 [==============================] - 1s 648us/step - loss: 5.7946e-04 0s - loss: 5.4
Epoch 427/500
2114/2114 [==============================] - 1s 645us/step - loss: 6.6615e-04
Epoch 428/500
2114/2114 [==============================] - 1s 652us/step - loss: 6.3112e-04
Epoch 429/500
2114/2114 [==============================] - 1s 685us/step - loss: 6.1640e-04
Epoch 430/500
2114/2114 [==============================] - 1s 691us/step - loss: 5.0742e-04
Epoch 431/500
2114/2114 [==============================] - 1s 668us/step - loss: 5.7810e-04
Epoch 432/500
2114/2114 [==============================] - 2s 711us/step - loss: 6.6876e-04
Epoch 433/500
2114/2114 [==============================] - 2s 893us/step - loss: 6.4580e-04
Epoch 434/500
2114/2114 [==============================] - 2s 728us/step - loss: 6.5976e-04
Epoch 435/500
2114/2114 [==============================] - 2s 741us/step - loss: 5.6575e-04
Epoch 436/500
2114/2114 [==============================] - 2s 734us/step - loss: 6.6602e-04
Epoch 437/500
2114/2114 [==============================] - 1s 707us/step - loss: 6.0253e-04
Epoch 438/500
2114/2114 [==============================] - 1s 696us/step - loss: 5.2597e-04
Epoch 439/500
2114/2114 [==============================] - 1s 687us/step - loss: 5.3381e-04
Epoch 440/500
2114/2114 [==============================] - 1s 683us/step - loss: 7.0305e-04
Epoch 441/500
2114/2114 [==============================] - 1s 655us/step - loss: 6.1364e-04
Epoch 442/500
2114/2114 [==============================] - 1s 634us/step - loss: 4.8041e-04 0s - l
Epoch 443/500
2114/2114 [==============================] - 2s 788us/step - loss: 6.0458e-04
Epoch 444/500
2114/2114 [==============================] - 2s 832us/step - loss: 5.5862e-04
Epoch 445/500
2114/2114 [==============================] - 1s 688us/step - loss: 7.0646e-04
Epoch 446/500
2114/2114 [==============================] - 1s 629us/step - loss: 5.4308e-04
Epoch 447/500
2114/2114 [==============================] - 1s 641us/step - loss: 6.2576e-04
Epoch 448/500
2114/2114 [==============================] - 1s 648us/step - loss: 5.5083e-04 0s - loss: 5.50
Epoch 449/500
2114/2114 [==============================] - 1s 632us/step - loss: 5.8677e-04
Epoch 450/500
2114/2114 [==============================] - 1s 656us/step - loss: 5.9713e-04
Epoch 451/500
2114/2114 [==============================] - 1s 685us/step - loss: 5.4976e-04
Epoch 452/500
2114/2114 [==============================] - 2s 712us/step - loss: 4.9254e-04
Epoch 453/500
2114/2114 [==============================] - 1s 702us/step - loss: 5.4101e-04
Epoch 454/500
2114/2114 [==============================] - 2s 804us/step - loss: 4.9872e-04
Epoch 455/500
2114/2114 [==============================] - 2s 781us/step - loss: 4.8403e-04
Epoch 456/500
2114/2114 [==============================] - 2s 713us/step - loss: 4.9325e-04
Epoch 457/500
2114/2114 [==============================] - 1s 662us/step - loss: 5.7780e-04
Epoch 458/500
2114/2114 [==============================] - 1s 664us/step - loss: 5.3627e-04
Epoch 459/500
2114/2114 [==============================] - 1s 670us/step - loss: 5.2742e-04
Epoch 460/500
2114/2114 [==============================] - 1s 636us/step - loss: 5.0896e-04
Epoch 461/500
2114/2114 [==============================] - 1s 621us/step - loss: 6.7138e-04
Epoch 462/500
2114/2114 [==============================] - 1s 683us/step - loss: 5.1272e-04
Epoch 463/500
2114/2114 [==============================] - 1s 657us/step - loss: 5.9007e-04
Epoch 464/500
2114/2114 [==============================] - 1s 622us/step - loss: 5.4225e-04
Epoch 465/500
2114/2114 [==============================] - 1s 635us/step - loss: 6.0678e-04 0s - loss: 6.3638
Epoch 466/500
2114/2114 [==============================] - 2s 815us/step - loss: 6.1269e-04
Epoch 467/500
2114/2114 [==============================] - 1s 698us/step - loss: 5.9724e-04
Epoch 468/500
2114/2114 [==============================] - 2s 729us/step - loss: 6.1807e-04
Epoch 469/500
2114/2114 [==============================] - 2s 725us/step - loss: 5.2630e-04
Epoch 470/500
2114/2114 [==============================] - 2s 755us/step - loss: 6.6079e-04
Epoch 471/500
2114/2114 [==============================] - 2s 758us/step - loss: 6.2403e-04
Epoch 472/500
2114/2114 [==============================] - 1s 700us/step - loss: 5.5914e-04
Epoch 473/500
2114/2114 [==============================] - 1s 692us/step - loss: 6.8727e-04
Epoch 474/500
2114/2114 [==============================] - 1s 678us/step - loss: 5.2621e-04
Epoch 475/500
2114/2114 [==============================] - 1s 679us/step - loss: 5.9292e-04
Epoch 476/500
2114/2114 [==============================] - 2s 905us/step - loss: 4.9175e-04
Epoch 477/500
2114/2114 [==============================] - 2s 717us/step - loss: 6.1136e-04
Epoch 478/500
2114/2114 [==============================] - 1s 699us/step - loss: 5.3795e-04
Epoch 479/500
2114/2114 [==============================] - 1s 652us/step - loss: 5.3394e-04 0s - loss: 5.0745e
Epoch 480/500
2114/2114 [==============================] - 1s 665us/step - loss: 7.2797e-04
Epoch 481/500
2114/2114 [==============================] - 1s 694us/step - loss: 5.5138e-04
Epoch 482/500
2114/2114 [==============================] - 1s 689us/step - loss: 5.2983e-04
Epoch 483/500
2114/2114 [==============================] - 1s 652us/step - loss: 5.2063e-04 1s 
Epoch 484/500
2114/2114 [==============================] - 1s 706us/step - loss: 6.2069e-04
Epoch 485/500
2114/2114 [==============================] - 2s 747us/step - loss: 5.0298e-04
Epoch 486/500
2114/2114 [==============================] - 2s 741us/step - loss: 5.2104e-04
Epoch 487/500
2114/2114 [==============================] - 2s 1ms/step - loss: 7.0094e-04
Epoch 488/500
2114/2114 [==============================] - 2s 763us/step - loss: 6.4404e-04
Epoch 489/500
2114/2114 [==============================] - 2s 901us/step - loss: 6.5419e-04
Epoch 490/500
2114/2114 [==============================] - 2s 763us/step - loss: 5.4483e-04
Epoch 491/500
2114/2114 [==============================] - 1s 707us/step - loss: 5.7262e-04
Epoch 492/500
2114/2114 [==============================] - 2s 745us/step - loss: 5.5717e-04
Epoch 493/500
2114/2114 [==============================] - 2s 908us/step - loss: 6.5252e-04
Epoch 494/500
2114/2114 [==============================] - 1s 670us/step - loss: 6.2740e-04
Epoch 495/500
2114/2114 [==============================] - 2s 882us/step - loss: 6.9801e-04
Epoch 496/500
2114/2114 [==============================] - 2s 977us/step - loss: 5.7754e-04
Epoch 497/500
2114/2114 [==============================] - 2s 749us/step - loss: 5.7544e-04
Epoch 498/500
2114/2114 [==============================] - 1s 681us/step - loss: 5.8934e-04
Epoch 499/500
2114/2114 [==============================] - 2s 711us/step - loss: 5.5475e-04
Epoch 500/500
2114/2114 [==============================] - 2s 713us/step - loss: 5.9717e-04

Loss function for RELU

In [38]:
pyplot.title('Mean Squared Error')
pyplot.plot(historyTwo.history['loss'], label='train')
pyplot.legend()
pyplot.show()
In [39]:
all_priceRulu = sigRulu.predict(X_train_3D)
In [40]:
All_inverseRulu_T =np.concatenate((X_train_4Nor, all_priceRulu), axis=1)
In [41]:
All_inverseRulu_T = pd.DataFrame(All_inverseRulu_T,columns=['0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15'])
All_inverseRulu_T.tail(5)
Out[41]:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
2109 0.008459 0.000521 3.606515e-06 0.489011 0.014703 0.0 0.101531 0.742215 0.000944 0.000000 0.022339 0.007341 0.000060 0.012552 0.000348 0.005671
2110 0.009717 0.000594 1.832925e-06 0.472527 0.027416 0.0 0.193424 0.726644 0.000992 0.006305 0.027228 0.008202 0.000047 0.014367 0.000402 0.006868
2111 0.004895 0.000302 8.352798e-07 0.357143 0.010709 0.0 0.110597 0.617647 0.000774 0.100000 0.026916 0.002917 0.000033 0.007157 0.000318 0.008878
2112 0.004622 0.000381 8.352798e-07 0.452381 0.010099 0.0 0.109903 0.707612 0.000745 0.100000 0.024805 0.003172 0.000010 0.007832 0.000309 0.009508
2113 0.006650 0.000570 0.000000e+00 0.410256 0.016235 0.0 0.126421 0.667820 0.001006 0.000000 0.026426 0.003908 0.000000 0.009226 0.000361 0.005477
In [42]:
columnsTitles = ['0','1','2','3','4','5','6','7','8','9','10','11','15','13','14','12']
All_inverseRulu_T=All_inverseRulu_T.reindex(columns=columnsTitles)
All_inverseRulu_T =All_inverseRulu_T[::-1]

All_inverseRulu_T.head(5)
Out[42]:
0 1 2 3 4 5 6 7 8 9 10 11 15 13 14 12
2113 0.006650 0.000570 0.000000e+00 0.410256 0.016235 0.0 0.126421 0.667820 0.001006 0.000000 0.026426 0.003908 0.005477 0.009226 0.000361 0.000000
2112 0.004622 0.000381 8.352798e-07 0.452381 0.010099 0.0 0.109903 0.707612 0.000745 0.100000 0.024805 0.003172 0.009508 0.007832 0.000309 0.000010
2111 0.004895 0.000302 8.352798e-07 0.357143 0.010709 0.0 0.110597 0.617647 0.000774 0.100000 0.026916 0.002917 0.008878 0.007157 0.000318 0.000033
2110 0.009717 0.000594 1.832925e-06 0.472527 0.027416 0.0 0.193424 0.726644 0.000992 0.006305 0.027228 0.008202 0.006868 0.014367 0.000402 0.000047
2109 0.008459 0.000521 3.606515e-06 0.489011 0.014703 0.0 0.101531 0.742215 0.000944 0.000000 0.022339 0.007341 0.005671 0.012552 0.000348 0.000060
In [43]:
All_inverseRulu_T = np.array(All_inverseRulu_T)
In [44]:
All_inverseRulu_T = trainMinMax.inverse_transform(All_inverseRulu_T)
In [45]:
All_inverseRulu_T
Out[45]:
array([[1.10500000e+04, 1.82345776e+06, 7.98626858e+02, ...,
        1.01005364e+08, 2.23377243e+03, 4.48450527e+05],
       [9.83200000e+03, 1.34492997e+06, 8.08548894e+02, ...,
        9.08518825e+07, 2.22220199e+03, 5.71106414e+05],
       [9.99600000e+03, 1.14548939e+06, 8.08548894e+02, ...,
        8.59378578e+07, 2.22424209e+03, 8.41514530e+05],
       ...,
       [7.06690000e+04, 8.83410322e+07, 1.18794958e+07, ...,
        7.44612429e+08, 5.74445943e+03, 7.39894557e+09],
       [9.98370000e+04, 9.13641667e+07, 1.18794958e+07, ...,
        7.80214701e+08, 5.94215548e+03, 7.40803421e+09],
       [1.02834000e+05, 2.72754401e+08, 1.18150952e+07, ...,
        8.26556232e+08, 1.17095829e+04, 7.42680941e+09]])
In [46]:
All_inverseRulu_T=All_inverseRulu_T[:,12]
All_inverseRulu_T = All_inverseRulu_T[::-1]

Two layer with Two sigmoid fuction.

In [47]:
regressor = Sequential()

#Adding the input layer and the LSTM layer
regressor.add(LSTM(units = 50, activation = 'sigmoid',return_sequences=True, input_shape = (None, 1)))
#regressor.add(LSTM(units=20,return_sequences=True))
#Adding the output layer
regressor.add(LSTM(units = 100, return_sequences=False))
regressor.add(Dropout(rate=.2))
regressor.add(Dense(units = 1))
#Compiling the Recurrent Neural Network
regressor.compile(optimizer = 'adam', loss = 'mean_squared_error')

Batch size 50 and epoch 500

In [48]:
#Fitting the Recurrent Neural Network [epoches is a kindoff number of iteration]

history= regressor.fit(X_train_3D, y_train, batch_size = 50, epochs = 500)
Epoch 1/500
2114/2114 [==============================] - 5s 2ms/step - loss: 0.0332
Epoch 2/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0240
Epoch 3/500
2114/2114 [==============================] - 5s 2ms/step - loss: 0.0233
Epoch 4/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0210
Epoch 5/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0188
Epoch 6/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0139
Epoch 7/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0125
Epoch 8/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0059
Epoch 9/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0047
Epoch 10/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0044
Epoch 11/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0038
Epoch 12/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0046
Epoch 13/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0035
Epoch 14/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0043
Epoch 15/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0039
Epoch 16/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0030
Epoch 17/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0032
Epoch 18/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0034
Epoch 19/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0034
Epoch 20/500
2114/2114 [==============================] - 2s 1ms/step - loss: 0.0034
Epoch 21/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0032
Epoch 22/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0031
Epoch 23/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0030
Epoch 24/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0028
Epoch 25/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0029
Epoch 26/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0028
Epoch 27/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0028
Epoch 28/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0030
Epoch 29/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0027
Epoch 30/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0027A: 0s - loss:
Epoch 31/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0026
Epoch 32/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0021
Epoch 33/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0020A: 3s - loss: 
Epoch 34/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0019A: 0s - 
Epoch 35/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0024A: 0s - loss: 0.
Epoch 36/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0021
Epoch 37/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0019
Epoch 38/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0023
Epoch 39/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0018
Epoch 40/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0018
Epoch 41/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0020
Epoch 42/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0019
Epoch 43/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0016A: 0s - lo
Epoch 44/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0016
Epoch 45/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0019
Epoch 46/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0016
Epoch 47/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0017
Epoch 48/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0027
Epoch 49/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0016
Epoch 50/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0014
Epoch 51/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0015
Epoch 52/500
2114/2114 [==============================] - 15s 7ms/step - loss: 0.0016
Epoch 53/500
2114/2114 [==============================] - 9s 4ms/step - loss: 0.0013
Epoch 54/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0016
Epoch 55/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0015A
Epoch 56/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0016
Epoch 57/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0015
Epoch 58/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0016
Epoch 59/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0013
Epoch 60/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0011
Epoch 61/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0011
Epoch 62/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0013
Epoch 63/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0011
Epoch 64/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0016
Epoch 65/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0012
Epoch 66/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0011
Epoch 67/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0011
Epoch 68/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0013
Epoch 69/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0015
Epoch 70/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0013
Epoch 71/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0012
Epoch 72/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.8073e-04
Epoch 73/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.8144e-04
Epoch 74/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0010TA: 1s - 
Epoch 75/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.7115e-04
Epoch 76/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0012
Epoch 77/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.9893e-04
Epoch 78/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0010
Epoch 79/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.7196e-04A:
Epoch 80/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0011
Epoch 81/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.0602e-04
Epoch 82/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.8873e-04
Epoch 83/500
2114/2114 [==============================] - 3s 1ms/step - loss: 0.0014
Epoch 84/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.7659e-04
Epoch 85/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0014A: 0s - loss: 0.001
Epoch 86/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0011
Epoch 87/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.4204e-04
Epoch 88/500
2114/2114 [==============================] - 3s 2ms/step - loss: 8.8479e-04
Epoch 89/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.0993e-04
Epoch 90/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.1352e-04
Epoch 91/500
2114/2114 [==============================] - 4s 2ms/step - loss: 0.0010
Epoch 92/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.0273e-04A: 0s - loss: 9.
Epoch 93/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.0127e-04
Epoch 94/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.1302e-04
Epoch 95/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.8116e-04
Epoch 96/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.0560e-04
Epoch 97/500
2114/2114 [==============================] - ETA: 0s - loss: 6.4673e-0 - 3s 2ms/step - loss: 6.4582e-04
Epoch 98/500
2114/2114 [==============================] - 4s 2ms/step - loss: 7.3751e-04
Epoch 99/500
2114/2114 [==============================] - 3s 1ms/step - loss: 8.0740e-04
Epoch 100/500
2114/2114 [==============================] - 3s 1ms/step - loss: 7.2674e-04
Epoch 101/500
2114/2114 [==============================] - 3s 2ms/step - loss: 7.7812e-04
Epoch 102/500
2114/2114 [==============================] - 4s 2ms/step - loss: 9.3670e-04
Epoch 103/500
2114/2114 [==============================] - 3s 2ms/step - loss: 0.0010
Epoch 104/500
2114/2114 [==============================] - 3s 2ms/step - loss: 8.6903e-04
Epoch 105/500
2114/2114 [==============================] - 4s 2ms/step - loss: 8.2241e-04
Epoch 106/500
2114/2114 [==============================] - 3s 2ms/step - loss: 6.3994e-04
Epoch 107/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.6839e-04
Epoch 108/500
2114/2114 [==============================] - 3s 1ms/step - loss: 6.6768e-04
Epoch 109/500
2114/2114 [==============================] - 3s 1ms/step - loss: 6.0203e-04
Epoch 110/500
2114/2114 [==============================] - 3s 1ms/step - loss: 6.6614e-04
Epoch 111/500
2114/2114 [==============================] - 3s 2ms/step - loss: 6.2047e-04 - ETA: 1s - los
Epoch 112/500
2114/2114 [==============================] - 4s 2ms/step - loss: 5.2014e-04A: 1s -
Epoch 113/500
2114/2114 [==============================] - 4s 2ms/step - loss: 4.7496e-04
Epoch 114/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.9196e-04
Epoch 115/500
2114/2114 [==============================] - 3s 2ms/step - loss: 6.4194e-04A: 1s - loss:
Epoch 116/500
2114/2114 [==============================] - 4s 2ms/step - loss: 5.1877e-04
Epoch 117/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.6777e-04
Epoch 118/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.5813e-04
Epoch 119/500
2114/2114 [==============================] - 3s 2ms/step - loss: 4.3185e-04
Epoch 120/500
2114/2114 [==============================] - 3s 2ms/step - loss: 5.0102e-04
Epoch 121/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.9666e-04
Epoch 122/500
2114/2114 [==============================] - 3s 2ms/step - loss: 4.4981e-04
Epoch 123/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.8278e-04
Epoch 124/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.8965e-04
Epoch 125/500
2114/2114 [==============================] - 3s 2ms/step - loss: 5.2412e-04
Epoch 126/500
2114/2114 [==============================] - 4s 2ms/step - loss: 6.8184e-04
Epoch 127/500
2114/2114 [==============================] - 4s 2ms/step - loss: 4.5531e-04
Epoch 128/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.2232e-04A: 0s - loss: 3.2120e
Epoch 129/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.5320e-04
Epoch 130/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.2330e-04
Epoch 131/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.0737e-04
Epoch 132/500
2114/2114 [==============================] - 3s 1ms/step - loss: 4.5491e-04A: 1s - l
Epoch 133/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.9946e-04
Epoch 134/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.6395e-04
Epoch 135/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.6918e-04
Epoch 136/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.3348e-04
Epoch 137/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.0086e-04
Epoch 138/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.3588e-04
Epoch 139/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.6628e-04
Epoch 140/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.8703e-04
Epoch 141/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.5010e-04
Epoch 142/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.7672e-04
Epoch 143/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.7065e-04
Epoch 144/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.8583e-04
Epoch 145/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.4091e-04
Epoch 146/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.1756e-04
Epoch 147/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.9079e-04
Epoch 148/500
2114/2114 [==============================] - 3s 2ms/step - loss: 5.7373e-04
Epoch 149/500
2114/2114 [==============================] - 4s 2ms/step - loss: 5.7969e-04A: 0s - loss: 6
Epoch 150/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.8725e-04
Epoch 151/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.9320e-04
Epoch 152/500
2114/2114 [==============================] - 2s 1ms/step - loss: 3.8690e-04
Epoch 153/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.5104e-04A: 1s
Epoch 154/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.3038e-04
Epoch 155/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.2158e-04
Epoch 156/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.6565e-04
Epoch 157/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.9428e-04
Epoch 158/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.9925e-04
Epoch 159/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.4497e-04
Epoch 160/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.9786e-04
Epoch 161/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.2130e-04
Epoch 162/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.0761e-04
Epoch 163/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.6918e-04A: 
Epoch 164/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.8629e-04
Epoch 165/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.9665e-04
Epoch 166/500
2114/2114 [==============================] - 3s 2ms/step - loss: 4.1024e-04
Epoch 167/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.8297e-04
Epoch 168/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.0585e-04
Epoch 169/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.9193e-04
Epoch 170/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.9254e-04
Epoch 171/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.7587e-04
Epoch 172/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.4907e-04
Epoch 173/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.1643e-04
Epoch 174/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.8697e-04A: 1s - lo
Epoch 175/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.2233e-04
Epoch 176/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.6807e-04
Epoch 177/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.7965e-04
Epoch 178/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.1536e-04
Epoch 179/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.9148e-04
Epoch 180/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.3736e-04
Epoch 181/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.6998e-04
Epoch 182/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.0852e-04
Epoch 183/500
2114/2114 [==============================] - 4s 2ms/step - loss: 5.0086e-04
Epoch 184/500
2114/2114 [==============================] - 3s 1ms/step - loss: 4.6225e-04
Epoch 185/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.7628e-04
Epoch 186/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.0614e-04
Epoch 187/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.1166e-04
Epoch 188/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.9110e-04
Epoch 189/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.4782e-04
Epoch 190/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.3357e-04
Epoch 191/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.1801e-04
Epoch 192/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.2420e-04
Epoch 193/500
2114/2114 [==============================] - 7s 3ms/step - loss: 2.7116e-04
Epoch 194/500
2114/2114 [==============================] - 7s 3ms/step - loss: 2.8263e-04
Epoch 195/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.0502e-04
Epoch 196/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.5821e-04
Epoch 197/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.5806e-04
Epoch 198/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.4687e-04
Epoch 199/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.2211e-04
Epoch 200/500
2114/2114 [==============================] - 2s 1ms/step - loss: 3.4345e-04
Epoch 201/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.0571e-04
Epoch 202/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8527e-04
Epoch 203/500
2114/2114 [==============================] - 2s 1ms/step - loss: 3.1930e-04
Epoch 204/500
2114/2114 [==============================] - 2s 1ms/step - loss: 3.1542e-04
Epoch 205/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.6399e-04
Epoch 206/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.2363e-04
Epoch 207/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.9999e-04
Epoch 208/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.3534e-04
Epoch 209/500
2114/2114 [==============================] - 2s 1ms/step - loss: 3.1945e-04
Epoch 210/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.2033e-04
Epoch 211/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.5831e-04
Epoch 212/500
2114/2114 [==============================] - 5s 3ms/step - loss: 3.0624e-04
Epoch 213/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.0979e-04
Epoch 214/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.8843e-04
Epoch 215/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.1653e-04
Epoch 216/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.0565e-04
Epoch 217/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.7755e-04
Epoch 218/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.1905e-04
Epoch 219/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.1880e-04
Epoch 220/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.6817e-04
Epoch 221/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.8623e-04
Epoch 222/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.8234e-04
Epoch 223/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.7086e-04
Epoch 224/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.5529e-04A: 3s - loss: 2.6
Epoch 225/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.6380e-04
Epoch 226/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.7268e-04
Epoch 227/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.8432e-04
Epoch 228/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.2966e-04
Epoch 229/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.8541e-04
Epoch 230/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.9515e-04
Epoch 231/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.0227e-04
Epoch 232/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.1986e-04
Epoch 233/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.9705e-04
Epoch 234/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.3719e-04
Epoch 235/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.0219e-04
Epoch 236/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.7045e-04A: 4s - loss:  
Epoch 237/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.8997e-04
Epoch 238/500
2114/2114 [==============================] - 6s 3ms/step - loss: 3.4467e-04
Epoch 239/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.6838e-04
Epoch 240/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.0472e-04
Epoch 241/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.6158e-04
Epoch 242/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.4511e-04
Epoch 243/500
2114/2114 [==============================] - 6s 3ms/step - loss: 3.2204e-04
Epoch 244/500
2114/2114 [==============================] - 5s 3ms/step - loss: 2.7988e-04
Epoch 245/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.8549e-04
Epoch 246/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.9265e-04
Epoch 247/500
2114/2114 [==============================] - 6s 3ms/step - loss: 2.8529e-04
Epoch 248/500
2114/2114 [==============================] - 6s 3ms/step - loss: 3.0551e-04
Epoch 249/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.3821e-04
Epoch 250/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.9251e-04
Epoch 251/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.9480e-04A: 0s - loss: 3
Epoch 252/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.7784e-04
Epoch 253/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.8314e-04
Epoch 254/500
2114/2114 [==============================] - 3s 2ms/step - loss: 4.0069e-04
Epoch 255/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.2687e-04
Epoch 256/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.2142e-04
Epoch 257/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.9411e-04
Epoch 258/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.0049e-04
Epoch 259/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.8948e-04
Epoch 260/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.7479e-04A: 0s - loss
Epoch 261/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.7872e-04
Epoch 262/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.5647e-04
Epoch 263/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.3781e-04
Epoch 264/500
2114/2114 [==============================] - 3s 2ms/step - loss: 3.2173e-04
Epoch 265/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.8702e-04
Epoch 266/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.8970e-04
Epoch 267/500
2114/2114 [==============================] - 5s 3ms/step - loss: 2.8849e-04
Epoch 268/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.8620e-04
Epoch 269/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.3141e-04
Epoch 270/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.4660e-04
Epoch 271/500
2114/2114 [==============================] - 6s 3ms/step - loss: 2.9489e-04
Epoch 272/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.7152e-04
Epoch 273/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.1383e-04
Epoch 274/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.4515e-04
Epoch 275/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.5165e-04
Epoch 276/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.5798e-04
Epoch 277/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.3588e-04
Epoch 278/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.4743e-04
Epoch 279/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.9206e-04
Epoch 280/500
2114/2114 [==============================] - 4s 2ms/step - loss: 3.0801e-04
Epoch 281/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.1022e-04
Epoch 282/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8033e-04
Epoch 283/500
2114/2114 [==============================] - 3s 1ms/step - loss: 3.8378e-04
Epoch 284/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.6908e-04
Epoch 285/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8205e-04
Epoch 286/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.6425e-04A: 0s - loss: 2.590
Epoch 287/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8372e-04
Epoch 288/500
2114/2114 [==============================] - 2s 932us/step - loss: 2.6256e-04
Epoch 289/500
2114/2114 [==============================] - 2s 998us/step - loss: 2.5096e-04
Epoch 290/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.5154e-04
Epoch 291/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8118e-04
Epoch 292/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.5928e-04
Epoch 293/500
2114/2114 [==============================] - 2s 978us/step - loss: 2.5442e-04
Epoch 294/500
2114/2114 [==============================] - 2s 779us/step - loss: 2.2614e-04
Epoch 295/500
2114/2114 [==============================] - 2s 981us/step - loss: 2.2088e-04 
Epoch 296/500
2114/2114 [==============================] - 2s 988us/step - loss: 2.1938e-04 0s - loss: 2.23
Epoch 297/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.6017e-04
Epoch 298/500
2114/2114 [==============================] - 2s 950us/step - loss: 2.3487e-04
Epoch 299/500
2114/2114 [==============================] - 2s 919us/step - loss: 2.4104e-04
Epoch 300/500
2114/2114 [==============================] - 2s 860us/step - loss: 2.3943e-04 2s
Epoch 301/500
2114/2114 [==============================] - 2s 929us/step - loss: 3.1226e-04
Epoch 302/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8960e-04
Epoch 303/500
2114/2114 [==============================] - 2s 796us/step - loss: 2.8266e-04
Epoch 304/500
2114/2114 [==============================] - 2s 980us/step - loss: 2.4914e-04
Epoch 305/500
2114/2114 [==============================] - 2s 857us/step - loss: 2.6700e-04
Epoch 306/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.9557e-04
Epoch 307/500
2114/2114 [==============================] - 2s 961us/step - loss: 2.2928e-04
Epoch 308/500
2114/2114 [==============================] - 2s 979us/step - loss: 2.7006e-04
Epoch 309/500
2114/2114 [==============================] - 2s 956us/step - loss: 2.2840e-04
Epoch 310/500
2114/2114 [==============================] - 2s 925us/step - loss: 2.0634e-04
Epoch 311/500
2114/2114 [==============================] - 2s 930us/step - loss: 2.7416e-04
Epoch 312/500
2114/2114 [==============================] - 2s 962us/step - loss: 2.6714e-04
Epoch 313/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.7127e-04
Epoch 314/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.6142e-04
Epoch 315/500
2114/2114 [==============================] - 2s 927us/step - loss: 2.1871e-04
Epoch 316/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.6571e-04
Epoch 317/500
2114/2114 [==============================] - 2s 930us/step - loss: 2.5915e-04
Epoch 318/500
2114/2114 [==============================] - 2s 963us/step - loss: 2.4707e-04
Epoch 319/500
2114/2114 [==============================] - 2s 972us/step - loss: 2.7873e-04
Epoch 320/500
2114/2114 [==============================] - 2s 928us/step - loss: 2.8909e-04
Epoch 321/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.2968e-04
Epoch 322/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.2197e-04
Epoch 323/500
2114/2114 [==============================] - 2s 827us/step - loss: 3.5325e-04
Epoch 324/500
2114/2114 [==============================] - 2s 864us/step - loss: 2.3837e-04
Epoch 325/500
2114/2114 [==============================] - 2s 923us/step - loss: 2.6844e-04 0s - loss: 2.740
Epoch 326/500
2114/2114 [==============================] - 2s 988us/step - loss: 2.5467e-04
Epoch 327/500
2114/2114 [==============================] - 2s 835us/step - loss: 2.9485e-04
Epoch 328/500
2114/2114 [==============================] - 2s 913us/step - loss: 2.5123e-04
Epoch 329/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.0608e-04
Epoch 330/500
2114/2114 [==============================] - 2s 898us/step - loss: 2.4486e-04
Epoch 331/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.3115e-04
Epoch 332/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.7032e-04
Epoch 333/500
2114/2114 [==============================] - 2s 850us/step - loss: 2.2934e-04
Epoch 334/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8474e-04
Epoch 335/500
2114/2114 [==============================] - 2s 853us/step - loss: 2.1039e-04
Epoch 336/500
2114/2114 [==============================] - 2s 844us/step - loss: 2.5646e-04
Epoch 337/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.3658e-04
Epoch 338/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8148e-04
Epoch 339/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.2052e-04
Epoch 340/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.3349e-04
Epoch 341/500
2114/2114 [==============================] - 2s 832us/step - loss: 2.5869e-04
Epoch 342/500
2114/2114 [==============================] - 2s 1ms/step - loss: 3.0411e-04
Epoch 343/500
2114/2114 [==============================] - 2s 858us/step - loss: 2.9290e-04
Epoch 344/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.3354e-04
Epoch 345/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.7532e-04
Epoch 346/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.8376e-04
Epoch 347/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.1489e-04
Epoch 348/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.3604e-04
Epoch 349/500
2114/2114 [==============================] - 2s 974us/step - loss: 2.5201e-04
Epoch 350/500
2114/2114 [==============================] - 2s 935us/step - loss: 3.3550e-04
Epoch 351/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.3469e-04
Epoch 352/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.1604e-04
Epoch 353/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.3515e-04
Epoch 354/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.2365e-04
Epoch 355/500
2114/2114 [==============================] - 2s 917us/step - loss: 2.2164e-04
Epoch 356/500
2114/2114 [==============================] - 2s 907us/step - loss: 2.3993e-04
Epoch 357/500
2114/2114 [==============================] - 2s 889us/step - loss: 2.1539e-04
Epoch 358/500
2114/2114 [==============================] - 2s 980us/step - loss: 2.3488e-04
Epoch 359/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.5768e-04
Epoch 360/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.5867e-04
Epoch 361/500
2114/2114 [==============================] - 2s 975us/step - loss: 2.0135e-04
Epoch 362/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.1045e-04
Epoch 363/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.5419e-04
Epoch 364/500
2114/2114 [==============================] - 2s 950us/step - loss: 2.2892e-04
Epoch 365/500
2114/2114 [==============================] - 2s 884us/step - loss: 2.5948e-04
Epoch 366/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.0719e-04
Epoch 367/500
2114/2114 [==============================] - 2s 966us/step - loss: 2.0663e-04
Epoch 368/500
2114/2114 [==============================] - 2s 1ms/step - loss: 3.5254e-04
Epoch 369/500
2114/2114 [==============================] - 2s 923us/step - loss: 2.4706e-04
Epoch 370/500
2114/2114 [==============================] - 2s 938us/step - loss: 2.6146e-04
Epoch 371/500
2114/2114 [==============================] - 2s 969us/step - loss: 2.2943e-04
Epoch 372/500
2114/2114 [==============================] - 2s 982us/step - loss: 2.3185e-04
Epoch 373/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.0513e-04
Epoch 374/500
2114/2114 [==============================] - 2s 1ms/step - loss: 1.9042e-04
Epoch 375/500
2114/2114 [==============================] - 2s 961us/step - loss: 1.7345e-04
Epoch 376/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.0329e-04
Epoch 377/500
2114/2114 [==============================] - 2s 977us/step - loss: 2.1797e-04
Epoch 378/500
2114/2114 [==============================] - 2s 873us/step - loss: 2.1467e-04
Epoch 379/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.5914e-04
Epoch 380/500
2114/2114 [==============================] - 2s 860us/step - loss: 2.3387e-04
Epoch 381/500
2114/2114 [==============================] - 2s 1ms/step - loss: 4.3513e-04
Epoch 382/500
2114/2114 [==============================] - 2s 981us/step - loss: 2.0482e-04
Epoch 383/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.6611e-04
Epoch 384/500
2114/2114 [==============================] - 2s 1ms/step - loss: 2.3282e-04
Epoch 385/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.3784e-04
Epoch 386/500
2114/2114 [==============================] - 6s 3ms/step - loss: 1.9638e-04
Epoch 387/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.0167e-04
Epoch 388/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.3254e-04
Epoch 389/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.1463e-04A:
Epoch 390/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.6562e-04
Epoch 391/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.0319e-04
Epoch 392/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.1231e-04
Epoch 393/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.3794e-04
Epoch 394/500
2114/2114 [==============================] - 5s 2ms/step - loss: 1.8611e-04
Epoch 395/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.6281e-04
Epoch 396/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.7721e-04
Epoch 397/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.8664e-04
Epoch 398/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.2489e-04
Epoch 399/500
2114/2114 [==============================] - 5s 2ms/step - loss: 3.0105e-04
Epoch 400/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.0035e-04
Epoch 401/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.1873e-04
Epoch 402/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.3437e-04
Epoch 403/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.9085e-04
Epoch 404/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.8782e-04
Epoch 405/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.3997e-04
Epoch 406/500
2114/2114 [==============================] - 5s 2ms/step - loss: 1.6991e-04
Epoch 407/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.8884e-04
Epoch 408/500
2114/2114 [==============================] - 5s 2ms/step - loss: 2.0921e-04
Epoch 409/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.9006e-04
Epoch 410/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.7638e-04
Epoch 411/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.0824e-04
Epoch 412/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.3161e-04
Epoch 413/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.4910e-04
Epoch 414/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.4452e-04
Epoch 415/500
2114/2114 [==============================] - ETA: 0s - loss: 2.1235e-0 - 3s 1ms/step - loss: 2.1165e-04
Epoch 416/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.4350e-04
Epoch 417/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7802e-04
Epoch 418/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.9123e-04
Epoch 419/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.4742e-04
Epoch 420/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.3027e-04
Epoch 421/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.2316e-04
Epoch 422/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7672e-04
Epoch 423/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.8839e-04
Epoch 424/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6283e-04
Epoch 425/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7652e-04
Epoch 426/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.5076e-04
Epoch 427/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6413e-04
Epoch 428/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.9054e-04
Epoch 429/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6916e-04
Epoch 430/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6085e-04
Epoch 431/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.0020e-04
Epoch 432/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.8471e-04
Epoch 433/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.5685e-04
Epoch 434/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.8666e-04
Epoch 435/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7218e-04
Epoch 436/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.7647e-04
Epoch 437/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.6141e-04
Epoch 438/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.6791e-04
Epoch 439/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.5391e-04
Epoch 440/500
2114/2114 [==============================] - 3s 2ms/step - loss: 2.2313e-04
Epoch 441/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.8015e-04
Epoch 442/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.0229e-04
Epoch 443/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.5514e-04
Epoch 444/500
2114/2114 [==============================] - 6s 3ms/step - loss: 1.9525e-04
Epoch 445/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.9675e-04
Epoch 446/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.8331e-04
Epoch 447/500
2114/2114 [==============================] - 6s 3ms/step - loss: 3.1877e-04A: 2
Epoch 448/500
2114/2114 [==============================] - 6s 3ms/step - loss: 2.0813e-04
Epoch 449/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.0973e-04
Epoch 450/500
2114/2114 [==============================] - 4s 2ms/step - loss: 2.0178e-04
Epoch 451/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.7174e-04
Epoch 452/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.7242e-04
Epoch 453/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.5808e-04
Epoch 454/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.1441e-04
Epoch 455/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.4371e-04
Epoch 456/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.0417e-04
Epoch 457/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.9976e-04
Epoch 458/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.0031e-04
Epoch 459/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7223e-04
Epoch 460/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.4881e-04
Epoch 461/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.3665e-04
Epoch 462/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.4583e-04
Epoch 463/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.3881e-04
Epoch 464/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.4664e-04
Epoch 465/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.3120e-04
Epoch 466/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.7737e-04
Epoch 467/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7387e-04
Epoch 468/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.9231e-04
Epoch 469/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.5010e-04
Epoch 470/500
2114/2114 [==============================] - 4s 2ms/step - loss: 1.8187e-04
Epoch 471/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6060e-04
Epoch 472/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7631e-04
Epoch 473/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.8053e-04A: 1
Epoch 474/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6587e-04A: 0s - loss: 1.6444e
Epoch 475/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6112e-04
Epoch 476/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7722e-04
Epoch 477/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6619e-04
Epoch 478/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.1198e-04
Epoch 479/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7401e-04
Epoch 480/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.3530e-04
Epoch 481/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.7237e-04
Epoch 482/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6373e-04
Epoch 483/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.5776e-04
Epoch 484/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.3569e-04
Epoch 485/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6030e-04
Epoch 486/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.6609e-04
Epoch 487/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.3351e-04
Epoch 488/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.4054e-04
Epoch 489/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6504e-04
Epoch 490/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6629e-04
Epoch 491/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.5744e-04
Epoch 492/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.0976e-04
Epoch 493/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.2540e-04
Epoch 494/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.5548e-04
Epoch 495/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.6888e-04
Epoch 496/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.7195e-04
Epoch 497/500
2114/2114 [==============================] - 3s 2ms/step - loss: 1.5940e-04
Epoch 498/500
2114/2114 [==============================] - 3s 1ms/step - loss: 2.4357e-04
Epoch 499/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.8632e-04
Epoch 500/500
2114/2114 [==============================] - 3s 1ms/step - loss: 1.2597e-04
In [49]:
regressor.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lstm_3 (LSTM)                (None, None, 50)          10400     
_________________________________________________________________
lstm_4 (LSTM)                (None, 100)               60400     
_________________________________________________________________
dropout_2 (Dropout)          (None, 100)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 101       
=================================================================
Total params: 70,901
Trainable params: 70,901
Non-trainable params: 0
_________________________________________________________________

Loss function for two sigmod

In [50]:
pyplot.title('Mean Squared Error')
pyplot.plot(history.history['loss'], label='train')
pyplot.legend()
pyplot.show()
In [51]:
all_price = regressor.predict(X_train_3D)
In [52]:
#nomal X_train + pred
All_inverse_T =np.concatenate((X_train_4Nor, all_price), axis=1)

Denormalization

In [53]:
All_inverse_T = pd.DataFrame(All_inverse_T,columns=['0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15'])
All_inverse_T.tail(5)
Out[53]:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
2109 0.008459 0.000521 3.606515e-06 0.489011 0.014703 0.0 0.101531 0.742215 0.000944 0.000000 0.022339 0.007341 0.000060 0.012552 0.000348 0.004059
2110 0.009717 0.000594 1.832925e-06 0.472527 0.027416 0.0 0.193424 0.726644 0.000992 0.006305 0.027228 0.008202 0.000047 0.014367 0.000402 0.007565
2111 0.004895 0.000302 8.352798e-07 0.357143 0.010709 0.0 0.110597 0.617647 0.000774 0.100000 0.026916 0.002917 0.000033 0.007157 0.000318 0.006417
2112 0.004622 0.000381 8.352798e-07 0.452381 0.010099 0.0 0.109903 0.707612 0.000745 0.100000 0.024805 0.003172 0.000010 0.007832 0.000309 0.007682
2113 0.006650 0.000570 0.000000e+00 0.410256 0.016235 0.0 0.126421 0.667820 0.001006 0.000000 0.026426 0.003908 0.000000 0.009226 0.000361 0.003041
In [54]:
columnsTitles = ['0','1','2','3','4','5','6','7','8','9','10','11','15','13','14','12']
All_inverse_T=All_inverse_T.reindex(columns=columnsTitles)
All_inverse_T =All_inverse_T[::-1]

All_inverse_T.head(5)
Out[54]:
0 1 2 3 4 5 6 7 8 9 10 11 15 13 14 12
2113 0.006650 0.000570 0.000000e+00 0.410256 0.016235 0.0 0.126421 0.667820 0.001006 0.000000 0.026426 0.003908 0.003041 0.009226 0.000361 0.000000
2112 0.004622 0.000381 8.352798e-07 0.452381 0.010099 0.0 0.109903 0.707612 0.000745 0.100000 0.024805 0.003172 0.007682 0.007832 0.000309 0.000010
2111 0.004895 0.000302 8.352798e-07 0.357143 0.010709 0.0 0.110597 0.617647 0.000774 0.100000 0.026916 0.002917 0.006417 0.007157 0.000318 0.000033
2110 0.009717 0.000594 1.832925e-06 0.472527 0.027416 0.0 0.193424 0.726644 0.000992 0.006305 0.027228 0.008202 0.007565 0.014367 0.000402 0.000047
2109 0.008459 0.000521 3.606515e-06 0.489011 0.014703 0.0 0.101531 0.742215 0.000944 0.000000 0.022339 0.007341 0.004059 0.012552 0.000348 0.000060
In [55]:
All_inverse_T = np.array(All_inverse_T)
In [56]:
All_inverse_T = trainMinMax.inverse_transform(All_inverse_T)
In [57]:
All_inverse_T
Out[57]:
array([[1.10500000e+04, 1.82345776e+06, 7.98626858e+02, ...,
        1.01005364e+08, 2.23377243e+03, 4.48450527e+05],
       [9.83200000e+03, 1.34492997e+06, 8.08548894e+02, ...,
        9.08518825e+07, 2.22220199e+03, 5.71106414e+05],
       [9.99600000e+03, 1.14548939e+06, 8.08548894e+02, ...,
        8.59378578e+07, 2.22424209e+03, 8.41514530e+05],
       ...,
       [7.06690000e+04, 8.83410322e+07, 1.18794958e+07, ...,
        7.44612429e+08, 5.74445943e+03, 7.39894557e+09],
       [9.98370000e+04, 9.13641667e+07, 1.18794958e+07, ...,
        7.80214701e+08, 5.94215548e+03, 7.40803421e+09],
       [1.02834000e+05, 2.72754401e+08, 1.18150952e+07, ...,
        8.26556232e+08, 1.17095829e+04, 7.42680941e+09]])
In [58]:
predicted_btc_price = regressor.predict(X_test_3D)

predicted_btc_price2 = model.predict(X_test_3D)

In [59]:
inverse_T =np.concatenate((X_test_4N, predicted_btc_price), axis=1)
In [60]:
inverse_T = pd.DataFrame(inverse_T,columns=['0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15'])
columnsTitles = ['0','1','2','3','4','5','6','7','8','9','10','11','15','13','14','12']
inverse_T=inverse_T.reindex(columns=columnsTitles)

inverse_T.head(5)
Out[60]:
0 1 2 3 4 5 6 7 8 9 10 11 15 13 14 12
0 0.427382 1.000000 0.986381 0.382353 0.483978 0.473046 0.172909 0.382353 0.922934 0.910517 0.385649 0.355475 1.090726 0.368126 1.000000 1.000000
1 0.398563 0.262565 1.000000 0.225490 0.372166 0.339485 0.120980 0.225490 0.890929 0.931034 0.277162 0.363200 0.946243 0.300381 0.330390 0.931427
2 0.118091 0.250274 1.000000 0.029412 0.031412 0.335017 0.145508 0.029412 0.943769 0.931034 0.439168 0.060428 0.797729 0.248335 0.307437 0.898232
3 0.791627 0.273466 0.976479 0.519608 1.000000 0.441205 0.191333 0.519608 0.998692 1.000000 0.411170 0.103414 1.022931 0.342198 0.419870 0.849767
4 1.000000 0.131751 0.902079 0.754902 0.471743 0.444882 0.094961 0.754902 1.000000 1.000000 0.316039 1.000000 0.907855 0.248763 0.305134 0.823203
In [61]:
#change to real number
inverse_T = testMinMax.inverse_transform(inverse_T)

dataset.head(10)
Out[61]:
activeAddresses adjustedTxVolume(USD) averageDifficulty blockCount blockSize exchangeVolume(USD) fees generatedCoins marketcap(USD) medianFee medianTxValue(USD) paymentCount price(USD) realizedCap(USD) txCount txVolume(USD)
0 102834.0 2.727544e+08 1.181510e+07 573.0 65641701.0 3.461536e+09 17.753439 14325.0 5.418036e+09 0.000221 75.837923 75246.0 88.39 4.632666e+09 26512.0 5.026659e+08
1 99837.0 9.136417e+07 1.187950e+07 557.0 53092124.0 2.765901e+09 16.292335 13925.0 5.317491e+09 0.000222 66.125885 76006.0 86.77 4.621039e+09 25088.0 1.995761e+08
2 70669.0 8.834103e+07 1.187950e+07 537.0 14846398.0 2.742631e+09 16.982480 13425.0 5.483492e+09 0.000222 80.628978 46219.0 89.50 4.615411e+09 23994.0 1.891868e+08
3 140714.0 9.404545e+07 1.176827e+07 587.0 123559210.0 3.295696e+09 18.271835 14675.0 5.656034e+09 0.000226 78.122568 50448.0 92.33 4.607194e+09 25967.0 2.400780e+08
4 162384.0 5.918720e+07 1.141644e+07 611.0 64268502.0 3.314849e+09 15.560253 15275.0 5.660144e+09 0.000226 69.606270 138655.0 92.42 4.602690e+09 24003.0 1.881442e+08
5 73882.0 7.553726e+07 1.141644e+07 599.0 14884957.0 3.857951e+09 15.615545 14975.0 5.431633e+09 0.000212 78.872088 49683.0 88.71 4.598286e+09 26033.0 2.290478e+08
6 77797.0 8.994418e+07 1.141644e+07 580.0 16160708.0 3.512862e+09 17.666264 14500.0 5.205337e+09 0.000192 77.999367 51084.0 85.04 4.587935e+09 26768.0 2.563719e+08
7 83650.0 1.552426e+08 1.079240e+07 588.0 18505439.0 4.195623e+09 19.729360 14700.0 5.212237e+09 0.000211 89.428500 53736.0 85.17 4.578644e+09 29731.0 3.004893e+08
8 105006.0 1.763392e+08 1.068695e+07 608.0 24619430.0 6.206102e+09 28.658120 15200.0 4.665896e+09 0.000224 130.835753 62947.0 76.26 4.572197e+09 39794.0 4.116363e+08
9 94751.0 1.680972e+08 1.068695e+07 630.0 24165146.0 4.161363e+09 30.859781 15750.0 3.713027e+09 0.000226 60.685189 59689.0 60.70 4.544131e+09 35034.0 4.544110e+08
In [62]:
inverse_T =pd.DataFrame(inverse_T)
In [63]:
print(inverse_T.shape)
inverse_T.head(5)
(60, 16)
Out[63]:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0 102834.0 2.727544e+08 1.181510e+07 573.0 65641701.0 3.461536e+09 17.753439 14325.0 5.418036e+09 0.000221 75.837923 75246.0 97.027085 4.525531e+09 39794.000000 5.026659e+08
1 99837.0 9.136417e+07 1.187950e+07 557.0 53092124.0 2.765901e+09 16.292335 13925.0 5.317491e+09 0.000222 66.125885 76006.0 89.690198 4.514045e+09 25718.805539 4.716270e+08
2 70669.0 8.834103e+07 1.187950e+07 537.0 14846398.0 2.742631e+09 16.982480 13425.0 5.483492e+09 0.000222 80.628978 46219.0 82.148698 4.505220e+09 25236.335600 4.566019e+08
3 140714.0 9.404545e+07 1.176827e+07 587.0 123559210.0 3.295696e+09 18.271835 14675.0 5.656034e+09 0.000226 78.122568 50448.0 93.584459 4.521135e+09 27599.674751 4.346648e+08
4 162384.0 5.918720e+07 1.141644e+07 611.0 64268502.0 3.314849e+09 15.560253 15275.0 5.660144e+09 0.000226 69.606270 138655.0 87.740894 4.505293e+09 25187.918283 4.226409e+08
In [64]:
inverse_T = np.array(inverse_T)
type(inverse_T)
Out[64]:
numpy.ndarray
In [65]:
inverse_T= inverse_T[:,12]
In [66]:
all_pred=All_inverse_T[:,12]
all_pred = all_pred[::-1]

MSE

In [67]:
#score for reulu
from sklearn.metrics import mean_squared_error
mean_squared_error(X_train_4_graph,All_inverseRulu_T)
Out[67]:
20.027729697685217
In [68]:
plt.figure(figsize=(50,50),dpi=80)
plt.plot(X_train_4_graph, color = 'red', label = 'Real LTC Value',linewidth=3, linestyle="--")
plt.plot(All_inverseRulu_T, color = 'green', label = 'Predicted LTC Value')
plt.title('litecoin Value Prediction' )
plt.xlabel('Days')
plt.ylabel('litecoin Value')
plt.legend()
plt.show()
In [69]:
#score $7.50 difference
from sklearn.metrics import mean_squared_error
mean_squared_error(X_train_4_graph,all_pred)
Out[69]:
7.492208642235717
In [70]:
plt.figure(figsize=(50,50),dpi=80)
plt.plot(X_train_4_graph, color = 'red', label = 'Real LTC Value',linewidth=3, linestyle="--")
plt.plot(all_pred, color = 'blue', label = 'Predicted LTC Value')
plt.title('litecoin Value Prediction' )
plt.xlabel('Days')
plt.ylabel('litecoin Value')
plt.legend()
plt.show()
In [71]:
plt.plot(y_test_4_graph, color = 'red', label = 'Real LTC Value' ,linewidth=.5, linestyle="--")
plt.plot(inverse_T, color = 'blue', label = 'Predicted LTC Value', linewidth=3)
plt.title('litecoin Value Prediction')
plt.xlabel('Days')
plt.ylabel('litecoin Value')
plt.legend()
plt.show()

In [ ]: